Commit Graph

100 Commits

Author SHA1 Message Date
ThePendulum 0d2bc9024f Stripping query for URL matching. 2023-06-05 04:16:48 +02:00
ThePendulum da8ffc0025 Fixed RedGIFs gallery support. 2023-06-05 04:11:34 +02:00
ThePendulum 2e923f2dcb Fixed YAML index file parser failing on duplicate keys. 2023-05-06 01:11:29 +02:00
ThePendulum 8a978cb803 Storing pHash. Fixed RedGIFs user agent. 2023-03-12 22:39:00 +01:00
ThePendulum 8704fb738b Using Imgur API rate limit feedback to prevent exceeding it. 2023-03-04 01:38:49 +01:00
ThePendulum c69ac9876a Added predata step. Using RedGifs temporary API key. 2023-02-27 02:37:08 +01:00
ThePendulum 57cac468df Fixed RedGIFs module content type. 2022-09-30 01:41:40 +02:00
ThePendulum 35ca8ccdd1 Fixed username in save message. 2021-12-18 00:40:12 +01:00
ThePendulum 66e5bfa50a Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs. 2021-12-17 02:04:25 +01:00
ThePendulum 196f318316 Cleanup 2019-12-22 03:20:24 +01:00
ThePendulum e1cc615a99 Added support for various tube and social media sites via youtube-dl. 2019-11-12 03:38:26 +01:00
ThePendulum 7de8f94caa Added hash comparison to duplicate avoidance. 2019-11-05 04:52:17 +01:00
ThePendulum e175ce38c8 Fetching and storing comments for self posts. 2019-11-05 01:58:00 +01:00
ThePendulum dddf4193b6 Removed post.user variable. 2019-11-02 02:46:11 +01:00
ThePendulum a7135f48a6 Added dynamic dividers to patterns. Fixed PornHub module. Updated README. 2019-11-02 02:19:23 +01:00
ThePendulum 401a4e2153 Completed Erome module. Added content URL list fetching. 2019-11-01 05:55:55 +01:00
ThePendulum eee3ba0abc Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules. 2019-11-01 04:22:36 +01:00
ThePendulum 4d87d94762 Removed debug log. Updated README to mention imgur API key requirement. 2019-10-31 05:24:38 +01:00
Niels Simenon db6317e560 Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file. 2019-10-31 05:22:07 +01:00
ThePendulum 727a25e61b Fixed gfycat fetching. 2018-12-26 15:47:45 +01:00
ThePendulum 73504a6d32 Added post score to index and variables. 2018-07-08 18:48:14 +02:00
ThePendulum 1b29d8fb96 Added basic PornHub support. 2018-07-08 01:08:20 +02:00
ThePendulum aa1c612306 Added after/before-index alias for after/before-indexed 2018-07-08 00:18:51 +02:00
ThePendulum 3abc6bb389 Always returning null for profile details when no profile was found, not undefined. Prevents YAML index write from failing. 2018-07-07 23:45:43 +02:00
ThePendulum 3a86f61835 Defaulting to jpeg if reddit image extension is not present. 2018-07-07 04:12:16 +02:00
ThePendulum f025aaa370 Added optional URL query to all external hosts in link dissection. 2018-07-07 04:07:07 +02:00
ThePendulum 3c9675ea85 Removed duplicate post link from imgur logs 2018-07-07 04:03:42 +02:00
ThePendulum d9ce4dd056 Added option to index (specified) ignored posts. Saving index through save module, and it now notifies the user that an index has been written. 2018-07-07 02:17:41 +02:00
ThePendulum 783e2d8765 Added byte truncate script to repo. Fixed yargs not enforcing choices. 2018-07-07 00:58:15 +02:00
ThePendulum 6c15d2e88b Calculating filename component limit in bytes rather than characters, preventing 'filename too long' errors when e.g. emojis are used 2018-07-07 00:37:10 +02:00
ThePendulum 4e19c25b0d Reintroduced imgur API as a fallback method. 2018-07-06 23:05:58 +02:00
ThePendulum a0e01c8837 Allowing for -mobile gfycat links, simplified URL patterns 2018-07-06 00:39:43 +02:00
ThePendulum 160ee9711d Accounting for actualid_d.jpg imgur pattern. Added debug argument to print full stack trace on error. 2018-07-05 23:27:11 +02:00
ThePendulum 2cc15eecd0 No longer relying on imgur API for albums. Removed config for imgur API. Fixed ?#0 URLs not being detected for imgur. 2018-07-05 02:54:29 +02:00
ThePendulum f5382ddc37 No longer using imgur API for individual images. Only saving EXIF data to JPEGs. Always using global exiftool instance. 2018-07-05 02:00:45 +02:00
ThePendulum 5dce74588b Improved missing gfycat handling. Added 'preview' marker to index. 2018-07-02 23:15:06 +02:00
ThePendulum 310931e4f8 Don't attempt to save index when there are no posts or profile details (complained about directory not existing) 2018-07-02 19:55:21 +02:00
ThePendulum c1f26440fd Added fetch concurrency. 2018-07-02 03:45:20 +02:00
ThePendulum b4356e8e11 Added support for file with host IDs to ignore. 2018-07-02 02:33:34 +02:00
ThePendulum 7335db1593 Allow usernames and post IDs to be read from file. 2018-07-02 00:25:48 +02:00
ThePendulum 5b023adf30 Switched from setTimeout to crontab for watch-mode. 2018-07-01 23:16:03 +02:00
ThePendulum 9ded9a131e Added timestamp to watch mode status log. 2018-07-01 03:38:29 +02:00
ThePendulum cb6fd0c7f6 Fixed profile image checking for indexed description instead of image. 2018-07-01 03:32:53 +02:00
ThePendulum 1b1323dc3d No longer redownload profile image and description when indexed, unless --redownload-profile is specified. 2018-07-01 03:26:26 +02:00
ThePendulum e7eef86b23 Added watch-mode. Waiting for profile detail write to finalize before new watch cycle and capture details in index file. 2018-07-01 03:06:57 +02:00
ThePendulum f9f9c8a0bd Fixed extension getting cut off at filename limit. Cleaned up save module. 2018-07-01 01:49:59 +02:00
ThePendulum c639503667 Fixed index file for single post fetching. 2018-07-01 01:07:32 +02:00
ThePendulum 240146e554 Added redownload option to ignore index file. 2018-07-01 00:22:17 +02:00
ThePendulum 84f7757712 Added date and indexed oldest/latest limit arguments. 2018-07-01 00:07:12 +02:00
ThePendulum a859ac573c Accounting for indexed posts. 2018-06-30 21:47:43 +02:00