39 Commits

Author SHA1 Message Date
3573f6ff5a
feat(cli): add page argument to start scraping from a specific page 2024-10-20 00:16:22 -04:00
ee0e938782 Update README.md 2024-10-20 05:59:14 +02:00
ca6df90460 Merge pull request 'better imgae' (#7) from ErrorNoInternet/r34-scraper:main into main
👍
2024-10-20 05:52:20 +02:00
325730bd37
better imgae 2024-10-19 23:50:08 -04:00
danmax
188b714741 added chunked downloads, added progress bars, cleaned up javalsai's bad code
Co-authored-by: ErrorNoInternet <errornointernet@envs.net>
2024-10-19 23:39:13 -04:00
danmax
b5a70e3426 added a function to download images to current directory 2024-10-19 22:24:30 -04:00
137378beb3 Merge pull request 'javalsai-changes' (#6) from javalsai-changes into main
Reviewed-on: https://git.javalsai.dynv6.net/danmax/r34-scraper/pulls/6
2024-10-19 21:39:40 +02:00
9c55ef19b8
add: license 2024-10-19 21:39:02 +02:00
022352b554
feat: misc improvements 2024-10-19 21:37:31 +02:00
ace4ac2811
fix: conflicts & feat: multiline logger 2024-10-19 21:13:02 +02:00
af16bdbdfe
fix merge conflicts (again) 2024-10-19 20:31:39 +02:00
063acc97db
merge master & misc feats 2024-10-19 20:26:25 +02:00
danmax
6831aaf6ff Revert "Merge pull request 'Update gitignore, add taap, luv gaylord' (#5) from UwU/not-r34:main into main"
This reverts commit 9744c81ef58d3d81215a2a70a3daab79b021b9e6, reversing
changes made to eeee4f50b36a14a5a11b1204b031909ded3a9918.
2024-10-19 13:24:44 -04:00
9744c81ef5 Merge pull request 'Update gitignore, add taap, luv gaylord' (#5) from UwU/not-r34:main into main
Reviewed-on: https://git.javalsai.dynv6.net/danmax/r34-scraper/pulls/5
2024-10-19 17:51:56 +02:00
GayLord
c1d67c9d84 Update epilog cause it didnt offend anyone 2024-10-19 17:50:05 +02:00
GayLord
89fca9d0a3 Update gitignore, add taap, luv gaylord 2024-10-19 14:05:30 +02:00
danmax
eeee4f50b3 added multithreading
Co-authored-by: ErrorNoInternet <errornointernet@envs.net>
2024-10-18 19:36:25 -04:00
abd91a6e95 Merge pull request 'chore(README): add example usage image' (#4) from grialion/r34-scraper:main into main
Reviewed-on: https://git.javalsai.dynv6.net/danmax/r34-scraper/pulls/4
2024-10-18 23:03:57 +02:00
grialion
1a5fc75162 chore(README): add example usage image 2024-10-18 23:00:26 +02:00
e5e586ca2a
Merge branch 'main' into javalsai-changes 2024-10-18 23:00:05 +02:00
722df4305d Merge pull request 'fix: img regex' (#3) from grialion/r34-scraper:main into main
Reviewed-on: https://git.javalsai.dynv6.net/danmax/r34-scraper/pulls/3
2024-10-18 22:56:07 +02:00
grialion
91eff584cb fix: img regex 2024-10-18 22:53:39 +02:00
5ce292d1c2
chore: modularity & good code 2024-10-18 19:26:11 +02:00
e62d2cc186
feat: add arg parsing 2024-10-18 18:50:40 +02:00
4acaf0308c
chore: define rust channel (+ fmt) 2024-10-18 18:43:18 +02:00
danmax
815c08cc36 fixed page pids 2024-10-16 23:48:04 -04:00
danmax
83f2bece96 updated regex 2024-10-16 23:39:59 -04:00
danmax
7e732c4a75 added exponential backoff 2024-10-16 23:20:24 -04:00
c954cdeaea Merge pull request 'refactor: tidy main.rs' (#1) from ErrorNoInternet/r34-scraper:main into main
Reviewed-on: https://git.javalsai.dynv6.net/danmax/r34-scraper/pulls/1
2024-10-16 04:08:15 +02:00
67e388bf81
chore(gitignore): ignore /target folder 2024-10-15 18:49:58 -04:00
9931bd8eaa
refactor: tidy main.rs 2024-10-15 18:49:36 -04:00
75ad148f55
chore: add lock file 2024-10-15 18:47:01 -04:00
13a626a943 Update README.md 2024-10-16 00:03:43 +02:00
danmax
da3575e905 fixed regex, now accepts formats with number on them 2024-10-15 01:02:33 -04:00
danmax
4116d8a8d4 added tag selector 2024-10-15 00:35:29 -04:00
danmax
36c0bfb0fe made scraper stop once there are no post urls 2024-10-15 00:20:24 -04:00
danmax
7b73c942b8 made scraper scrape all the pages 2024-10-14 23:59:06 -04:00
danmax
91568abea5 added a readme 2024-10-14 22:36:13 -04:00
danmax
6dd45a17fa done with basics 2024-10-14 22:09:05 -04:00