Commit Graph

45 Commits

Author SHA1 Message Date
ba922bb385 Fixed .false file extension. 2026-08-17 23:31:29 +02:00
941cb280be Added proxy. 2026-08-17 22:48:02 +02:00
5c7a0397c9 Bumped to Node 24 and upgraded dependencies. 2026-08-09 02:03:54 +02:00
2e4ba27d98 Replaced Snoowrap with public API calls in response to Reddit revoking API keys. This was largely done using Claude Code, extensive manual review and major version bump needed if merged into main. 2026-08-07 23:37:26 +02:00
5923bb00bc Updated Node to v20 (blake2 incompatible with newer versions), replaced youtube-dl with youtube-dl-exec. 2025-11-28 01:15:59 +01:00
e642203d6c Fixed invalid URL breaking scrape, prefixing reddit URLs with origin. 2024-09-11 05:16:59 +02:00
d1ac6d0068 Fixed RedGIFs gallery support. 2024-09-11 05:16:59 +02:00
afaa428fec Fixed YAML index file parser failing on duplicate keys. 2024-09-11 05:16:58 +02:00
b7dc3f5fd1 Storing pHash. Fixed RedGIFs user agent. 2024-09-11 05:16:58 +02:00
7d633c31b4 Using Imgur API rate limit feedback to prevent exceeding it. 2024-09-11 05:16:58 +02:00
bed4fe288f Added predata step. Using RedGifs temporary API key. 2024-09-11 05:16:58 +02:00
529f84d9c2 Fixed RedGIFs module content type. 2024-09-11 05:16:58 +02:00
de50d609f3 Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs. 2024-09-11 05:16:58 +02:00
bb06fe9763 Cleanup 2024-09-11 05:16:58 +02:00
6e3bfdd175 Added support for various tube and social media sites via youtube-dl. 2024-09-11 05:16:58 +02:00
7de0c51c16 Fetching and storing comments for self posts. 2024-09-11 05:16:58 +02:00
9140b58c70 Removed post.user variable. 2024-09-11 05:16:58 +02:00
5c7dca19e5 Added dynamic dividers to patterns. Fixed PornHub module. Updated README. 2024-09-11 05:16:57 +02:00
443f0fc028 Completed Erome module. Added content URL list fetching. 2024-09-11 05:16:57 +02:00
b9a7e4b83a Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules. 2024-09-11 05:16:57 +02:00
416a3254fb Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file. 2024-09-11 05:16:57 +02:00
c21ca0df0c Fixed gfycat fetching. 2024-09-11 05:16:57 +02:00
8d75cc170d Added basic PornHub support. 2024-09-11 05:16:57 +02:00
246ed1576f Defaulting to jpeg if reddit image extension is not present. 2024-09-11 05:16:57 +02:00
82b7f38336 Removed duplicate post link from imgur logs 2024-09-11 05:16:57 +02:00
fab6d0aa1c Reintroduced imgur API as a fallback method. 2024-09-11 05:16:57 +02:00
0377053d5a Allowing for -mobile gfycat links, simplified URL patterns 2024-09-11 05:16:56 +02:00
91550406fe Accounting for actualid_d.jpg imgur pattern. Added debug argument to print full stack trace on error. 2024-09-11 05:16:56 +02:00
e950a9ac54 No longer relying on imgur API for albums. Removed config for imgur API. Fixed ?#0 URLs not being detected for imgur. 2024-09-11 05:16:56 +02:00
dbc4c45601 No longer using imgur API for individual images. Only saving EXIF data to JPEGs. Always using global exiftool instance. 2024-09-11 05:16:56 +02:00
b05ae06b00 Improved missing gfycat handling. Added 'preview' marker to index. 2024-09-11 05:16:56 +02:00
c39f2eda8c Improved handling of unavailable/deleted vidble videos 2024-09-11 05:16:55 +02:00
4e0fe3e6bc Added vidble direct video (/watch?v=) support. 2024-09-11 05:16:55 +02:00
e0ca4a0822 Added new eroshare method source files. 2024-09-11 05:16:55 +02:00
583b567b20 Added support for /i/ eroshare items. 2024-09-11 05:16:55 +02:00
5165e8612d Added vidble image support. 2024-09-11 05:16:55 +02:00
89ab431a2f Added support for Vidble (image) albums. Replaced extensions.json and reverseLookup with mime-types lib. 2024-09-11 05:16:55 +02:00
60eeca09d8 Added support for m.imgur.com links. Changed curate ignore log message details. 2024-09-11 05:16:55 +02:00
7f00e5b6b2 Added support for reddit preview fallback. 2024-09-11 05:16:54 +02:00
f0e312f2b1 Added archive support, and the IP archive. 2024-09-11 05:16:54 +02:00
2d050be1a5 Added support for redd.it PNG. 2024-09-11 05:16:54 +02:00
1581e780b8 Checking result status rather than ok for imgur methods. 2024-09-11 05:16:54 +02:00
4af6aa85d3 Fixed obsolete eroshare extract single album item property. Throwing error when imgur album and image info fetch fails. Main catch method console log colored red. 2024-09-11 05:16:54 +02:00
536c427140 Added support for muxing streams, specifically for reddit-hosted videos (now also supported). 2024-09-11 05:16:54 +02:00
2b9801999b Moved source into dedicated directory. 2024-09-11 05:16:53 +02:00