 DebaucheryLibrarian
|
bed4fe288f
|
Added predata step. Using RedGifs temporary API key.
|
2024-09-11 05:16:58 +02:00 |
|
DebaucheryLibrarian
|
de50d609f3
|
Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs.
|
2024-09-11 05:16:58 +02:00 |
|
DebaucheryLibrarian
|
9140b58c70
|
Removed post.user variable.
|
2024-09-11 05:16:58 +02:00 |
|
DebaucheryLibrarian
|
8d72683ebc
|
Renamed from reddit-post-dump to ripunzel.
|
2024-09-11 05:16:57 +02:00 |
|
DebaucheryLibrarian
|
5c7dca19e5
|
Added dynamic dividers to patterns. Fixed PornHub module. Updated README.
|
2024-09-11 05:16:57 +02:00 |
|
DebaucheryLibrarian
|
b9a7e4b83a
|
Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules.
|
2024-09-11 05:16:57 +02:00 |
|
DebaucheryLibrarian
|
fab6d0aa1c
|
Reintroduced imgur API as a fallback method.
|
2024-09-11 05:16:57 +02:00 |
|
DebaucheryLibrarian
|
e950a9ac54
|
No longer relying on imgur API for albums. Removed config for imgur API. Fixed ?#0 URLs not being detected for imgur.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
18acb3a878
|
Added fetch concurrency.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
00de0d43f5
|
Switched from setTimeout to crontab for watch-mode.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
fcb85f57c8
|
Added watch-mode. Waiting for profile detail write to finalize before new watch cycle and capture details in index file.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
029351f228
|
Using YAML rather than TSV for index files. Improves both readability and reindexability.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
c49e1edca0
|
Reading index file and ignoring already indexed content.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
952392e0d9
|
Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist).
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
5eb2eb651a
|
Saving index per user as TSV. Refactoring.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
bba0f8f1d7
|
Fetching and curating posts per user without merge.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
be7bc1b10d
|
Added filepath component length limit with truncator. Fixed double extension dot when copying original extension. Moved API keys to local config.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
89ab431a2f
|
Added support for Vidble (image) albums. Replaced extensions.json and reverseLookup with mime-types lib.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
e01979a757
|
Added include and exclude source arguments. Improved stream fetch failure handling and improved logging details.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
7f00e5b6b2
|
Added support for reddit preview fallback.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
f0e312f2b1
|
Added archive support, and the IP archive.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
8a57a9a8f9
|
Added reddit avatar URL detection and avoid option.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
07d157604d
|
Saving EXIF data for jpegs.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
7e6828bd63
|
Added support for reddituploads.com. Changed default sorting to 'new'.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
901029d9a1
|
Changed ignoreDuplicates to avoidDuplicates.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
df3421639f
|
Flow and modularization refactor. Added duplicates option and applying limit after fetch.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
1b0bd3a5a4
|
Refactoring flow. Added user and profile saving and variables.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
c3fc11d1ea
|
Added single item album extraction with configurable display variable, as well as a host/source name variable.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
a6ff72ae0c
|
Added groundwork for Eroshare support. Removed gfycat API requirement, using JSON output.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
9cde1ce31a
|
Added default config to repository.
|
2024-09-11 05:16:53 +02:00 |