|
|
b7dc3f5fd1
|
Storing pHash. Fixed RedGIFs user agent.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
7d633c31b4
|
Using Imgur API rate limit feedback to prevent exceeding it.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
bed4fe288f
|
Added predata step. Using RedGifs temporary API key.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
529f84d9c2
|
Fixed RedGIFs module content type.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
350a2dc1dc
|
Fixed username in save message.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
de50d609f3
|
Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
bb06fe9763
|
Cleanup
|
2024-09-11 05:16:58 +02:00 |
|
|
|
6e3bfdd175
|
Added support for various tube and social media sites via youtube-dl.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
45123bc630
|
Added hash comparison to duplicate avoidance.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
7de0c51c16
|
Fetching and storing comments for self posts.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
9140b58c70
|
Removed post.user variable.
|
2024-09-11 05:16:58 +02:00 |
|
|
|
5c7dca19e5
|
Added dynamic dividers to patterns. Fixed PornHub module. Updated README.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
443f0fc028
|
Completed Erome module. Added content URL list fetching.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
b9a7e4b83a
|
Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
8b85ec219d
|
Removed debug log. Updated README to mention imgur API key requirement.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
416a3254fb
|
Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
c21ca0df0c
|
Fixed gfycat fetching.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
60973a0e70
|
Added post score to index and variables.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
8d75cc170d
|
Added basic PornHub support.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
9b0ef4da91
|
Added after/before-index alias for after/before-indexed
|
2024-09-11 05:16:57 +02:00 |
|
|
|
a817a07250
|
Always returning null for profile details when no profile was found, not undefined. Prevents YAML index write from failing.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
246ed1576f
|
Defaulting to jpeg if reddit image extension is not present.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
db9f6ed60c
|
Added optional URL query to all external hosts in link dissection.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
82b7f38336
|
Removed duplicate post link from imgur logs
|
2024-09-11 05:16:57 +02:00 |
|
|
|
e48c00b529
|
Added option to index (specified) ignored posts. Saving index through save module, and it now notifies the user that an index has been written.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
c30006f361
|
Added byte truncate script to repo. Fixed yargs not enforcing choices.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
12e17c7c3c
|
Calculating filename component limit in bytes rather than characters, preventing 'filename too long' errors when e.g. emojis are used
|
2024-09-11 05:16:57 +02:00 |
|
|
|
fab6d0aa1c
|
Reintroduced imgur API as a fallback method.
|
2024-09-11 05:16:57 +02:00 |
|
|
|
0377053d5a
|
Allowing for -mobile gfycat links, simplified URL patterns
|
2024-09-11 05:16:56 +02:00 |
|
|
|
91550406fe
|
Accounting for actualid_d.jpg imgur pattern. Added debug argument to print full stack trace on error.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
e950a9ac54
|
No longer relying on imgur API for albums. Removed config for imgur API. Fixed ?#0 URLs not being detected for imgur.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
dbc4c45601
|
No longer using imgur API for individual images. Only saving EXIF data to JPEGs. Always using global exiftool instance.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
b05ae06b00
|
Improved missing gfycat handling. Added 'preview' marker to index.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
a68493d0fa
|
Don't attempt to save index when there are no posts or profile details (complained about directory not existing)
|
2024-09-11 05:16:56 +02:00 |
|
|
|
18acb3a878
|
Added fetch concurrency.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
91577a00b1
|
Added support for file with host IDs to ignore.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
352b2a66d7
|
Allow usernames and post IDs to be read from file.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
00de0d43f5
|
Switched from setTimeout to crontab for watch-mode.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
c4afd398dc
|
Added timestamp to watch mode status log.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
438ba05260
|
Fixed profile image checking for indexed description instead of image.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
e087e47069
|
No longer redownload profile image and description when indexed, unless --redownload-profile is specified.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
fcb85f57c8
|
Added watch-mode. Waiting for profile detail write to finalize before new watch cycle and capture details in index file.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
dcf7fdd274
|
Fixed extension getting cut off at filename limit. Cleaned up save module.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
99c7d143f7
|
Fixed index file for single post fetching.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
cdd5ed36bf
|
Added redownload option to ignore index file.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
24a165cf48
|
Added date and indexed oldest/latest limit arguments.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
86002ef00b
|
Accounting for indexed posts.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
029351f228
|
Using YAML rather than TSV for index files. Improves both readability and reindexability.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
f41b788183
|
Extracting indexed posts from fetched posts to allow for index rewrite flexibility.
|
2024-09-11 05:16:56 +02:00 |
|
|
|
c49e1edca0
|
Reading index file and ignoring already indexed content.
|
2024-09-11 05:16:56 +02:00 |
|
