Commit Graph

40 Commits

Author SHA1 Message Date
e642203d6c Fixed invalid URL breaking scrape, prefixing reddit URLs with origin. 2024-09-11 05:16:59 +02:00
d1ac6d0068 Fixed RedGIFs gallery support. 2024-09-11 05:16:59 +02:00
afaa428fec Fixed YAML index file parser failing on duplicate keys. 2024-09-11 05:16:58 +02:00
b7dc3f5fd1 Storing pHash. Fixed RedGIFs user agent. 2024-09-11 05:16:58 +02:00
7d633c31b4 Using Imgur API rate limit feedback to prevent exceeding it. 2024-09-11 05:16:58 +02:00
bed4fe288f Added predata step. Using RedGifs temporary API key. 2024-09-11 05:16:58 +02:00
529f84d9c2 Fixed RedGIFs module content type. 2024-09-11 05:16:58 +02:00
de50d609f3 Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs. 2024-09-11 05:16:58 +02:00
bb06fe9763 Cleanup 2024-09-11 05:16:58 +02:00
6e3bfdd175 Added support for various tube and social media sites via youtube-dl. 2024-09-11 05:16:58 +02:00
7de0c51c16 Fetching and storing comments for self posts. 2024-09-11 05:16:58 +02:00
9140b58c70 Removed post.user variable. 2024-09-11 05:16:58 +02:00
5c7dca19e5 Added dynamic dividers to patterns. Fixed PornHub module. Updated README. 2024-09-11 05:16:57 +02:00
443f0fc028 Completed Erome module. Added content URL list fetching. 2024-09-11 05:16:57 +02:00
b9a7e4b83a Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules. 2024-09-11 05:16:57 +02:00
416a3254fb Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file. 2024-09-11 05:16:57 +02:00
c21ca0df0c Fixed gfycat fetching. 2024-09-11 05:16:57 +02:00
8d75cc170d Added basic PornHub support. 2024-09-11 05:16:57 +02:00
246ed1576f Defaulting to jpeg if reddit image extension is not present. 2024-09-11 05:16:57 +02:00
82b7f38336 Removed duplicate post link from imgur logs 2024-09-11 05:16:57 +02:00
fab6d0aa1c Reintroduced imgur API as a fallback method. 2024-09-11 05:16:57 +02:00
0377053d5a Allowing for -mobile gfycat links, simplified URL patterns 2024-09-11 05:16:56 +02:00
91550406fe Accounting for actualid_d.jpg imgur pattern. Added debug argument to print full stack trace on error. 2024-09-11 05:16:56 +02:00
e950a9ac54 No longer relying on imgur API for albums. Removed config for imgur API. Fixed ?#0 URLs not being detected for imgur. 2024-09-11 05:16:56 +02:00
dbc4c45601 No longer using imgur API for individual images. Only saving EXIF data to JPEGs. Always using global exiftool instance. 2024-09-11 05:16:56 +02:00
b05ae06b00 Improved missing gfycat handling. Added 'preview' marker to index. 2024-09-11 05:16:56 +02:00
c39f2eda8c Improved handling of unavailable/deleted vidble videos 2024-09-11 05:16:55 +02:00
4e0fe3e6bc Added vidble direct video (/watch?v=) support. 2024-09-11 05:16:55 +02:00
e0ca4a0822 Added new eroshare method source files. 2024-09-11 05:16:55 +02:00
583b567b20 Added support for /i/ eroshare items. 2024-09-11 05:16:55 +02:00
5165e8612d Added vidble image support. 2024-09-11 05:16:55 +02:00
89ab431a2f Added support for Vidble (image) albums. Replaced extensions.json and reverseLookup with mime-types lib. 2024-09-11 05:16:55 +02:00
60eeca09d8 Added support for m.imgur.com links. Changed curate ignore log message details. 2024-09-11 05:16:55 +02:00
7f00e5b6b2 Added support for reddit preview fallback. 2024-09-11 05:16:54 +02:00
f0e312f2b1 Added archive support, and the IP archive. 2024-09-11 05:16:54 +02:00
2d050be1a5 Added support for redd.it PNG. 2024-09-11 05:16:54 +02:00
1581e780b8 Checking result status rather than ok for imgur methods. 2024-09-11 05:16:54 +02:00
4af6aa85d3 Fixed obsolete eroshare extract single album item property. Throwing error when imgur album and image info fetch fails. Main catch method console log colored red. 2024-09-11 05:16:54 +02:00
536c427140 Added support for muxing streams, specifically for reddit-hosted videos (now also supported). 2024-09-11 05:16:54 +02:00
2b9801999b Moved source into dedicated directory. 2024-09-11 05:16:53 +02:00