Commit Graph

119 Commits

Author SHA1 Message Date
DebaucheryLibrarian b7dc3f5fd1 Storing pHash. Fixed RedGIFs user agent. 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 3931996e1b 1.26.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 7ec4143972 1.25.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian bed4fe288f Added predata step. Using RedGifs temporary API key. 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 4acbe16fb8 1.24.2 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 7eaaadaa2a 1.24.1 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 7f79a44830 1.24.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian de50d609f3 Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs. 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 58a35fe998 1.23.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 17c7acd6a2 1.22.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 45123bc630 Added hash comparison to duplicate avoidance. 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 7c6c99c9d1 1.21.0 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 7de0c51c16 Fetching and storing comments for self posts. 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian 926ab1d25c 1.20.2 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian e1a57a07a3 1.20.1 2024-09-11 05:16:58 +02:00
DebaucheryLibrarian bcd5759d0f 1.20.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 66189694fa 1.19.3 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 54dd191d6c 1.19.2 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 0c50bcab32 1.19.1 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 50f1621a26 1.19.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 5c7dca19e5 Added dynamic dividers to patterns. Fixed PornHub module. Updated README. 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian c4d434ecdf 1.18.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian aecf79fa25 1.17.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian b9a7e4b83a Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules. 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 20cb522689 1.16.3 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian da90b832a1 1.16.2 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 416a3254fb Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file. 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian d5d56b83bd 1.16.1 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian d7916af3d5 1.16.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 12268fcb05 1.15.0 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian f47b99501a 1.14.9 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian bdd79031ef 1.14.8 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 7711979199 1.14.7 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian 34f7766496 1.14.6 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian b388142d8b 1.14.5 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian c1b164a5f9 1.14.4 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian e44f617d9e 1.14.3 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian e5d439cbaf 1.14.2 2024-09-11 05:16:57 +02:00
DebaucheryLibrarian e48892126f 1.14.1 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 44cdd37e49 1.14.0 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian e71b54e828 1.13.1 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian b5da928c98 1.13.0 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 00de0d43f5 Switched from setTimeout to crontab for watch-mode. 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian f989bc9f18 1.12.1 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian c83421ca41 1.12.0 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 029351f228 Using YAML rather than TSV for index files. Improves both readability and reindexability. 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 952392e0d9 Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist). 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 5eb2eb651a Saving index per user as TSV. Refactoring. 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 6696438da0 Refactored info content fetching and saving to handle users object and utilize async/await. 2024-09-11 05:16:56 +02:00
DebaucheryLibrarian 3c64ffcd45 1.11.2 2024-09-11 05:16:56 +02:00