Commit Graph

30 Commits

Author SHA1 Message Date
ThePendulum eee3ba0abc Added support for fetching content directly from host. Improved pattern interpolation. Refactored content modules. 2019-11-01 04:22:36 +01:00
ThePendulum 73504a6d32 Added post score to index and variables. 2018-07-08 18:48:14 +02:00
ThePendulum 3abc6bb389 Always returning null for profile details when no profile was found, not undefined. Prevents YAML index write from failing. 2018-07-07 23:45:43 +02:00
ThePendulum d9ce4dd056 Added option to index (specified) ignored posts. Saving index through save module, and it now notifies the user that an index has been written. 2018-07-07 02:17:41 +02:00
ThePendulum 6c15d2e88b Calculating filename component limit in bytes rather than characters, preventing 'filename too long' errors when e.g. emojis are used 2018-07-07 00:37:10 +02:00
ThePendulum f5382ddc37 No longer using imgur API for individual images. Only saving EXIF data to JPEGs. Always using global exiftool instance. 2018-07-05 02:00:45 +02:00
ThePendulum 5dce74588b Improved missing gfycat handling. Added 'preview' marker to index. 2018-07-02 23:15:06 +02:00
ThePendulum 310931e4f8 Don't attempt to save index when there are no posts or profile details (complained about directory not existing) 2018-07-02 19:55:21 +02:00
ThePendulum c1f26440fd Added fetch concurrency. 2018-07-02 03:45:20 +02:00
ThePendulum cb6fd0c7f6 Fixed profile image checking for indexed description instead of image. 2018-07-01 03:32:53 +02:00
ThePendulum 1b1323dc3d No longer redownload profile image and description when indexed, unless --redownload-profile is specified. 2018-07-01 03:26:26 +02:00
ThePendulum e7eef86b23 Added watch-mode. Waiting for profile detail write to finalize before new watch cycle and capture details in index file. 2018-07-01 03:06:57 +02:00
ThePendulum f9f9c8a0bd Fixed extension getting cut off at filename limit. Cleaned up save module. 2018-07-01 01:49:59 +02:00
ThePendulum c639503667 Fixed index file for single post fetching. 2018-07-01 01:07:32 +02:00
ThePendulum a859ac573c Accounting for indexed posts. 2018-06-30 21:47:43 +02:00
ThePendulum 74e36a6826 Using YAML rather than TSV for index files. Improves both readability and reindexability. 2018-06-30 03:33:30 +02:00
ThePendulum 33ef3d2c19 Reading index file and ignoring already indexed content. 2018-06-17 03:39:12 +02:00
ThePendulum 7cf1a99915 Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist). 2018-06-17 01:11:10 +02:00
ThePendulum 569213e291 Added filepath component length limit with truncator. Fixed double extension dot when copying original extension. Moved API keys to local config. 2018-05-23 17:03:12 +02:00
ThePendulum edf77e259b Refactored post fetching. Cleaner code that will now fetch archived posts even if the user no longer exists on reddit. Expanded gfycat URL detection. 2018-05-22 22:55:50 +02:00
ThePendulum 068ffbdbd8 Added include and exclude source arguments. Improved stream fetch failure handling and improved logging details. 2018-05-05 17:08:40 +02:00
ThePendulum 632dc96bc1 Fixed profile description not saving when user has avatar. 2018-04-30 14:17:44 +02:00
ThePendulum 9e9cca8834 Fixed avatar property. 2018-04-28 00:27:29 +02:00
ThePendulum ccfdc16ca8 Added reddit avatar URL detection and avoid option. 2018-04-27 23:14:48 +02:00
ThePendulum 88425aee5b Fixed double dir path component extraction in save module (causing profile info save to fail). 2018-04-27 00:42:11 +02:00
ThePendulum 0313b291da Adapted primary save function to handle single streams and non-item saves (e.g. profiles). 2018-04-26 03:35:42 +02:00
ThePendulum e752667be7 Added support for muxing streams, specifically for reddit-hosted videos (now also supported). 2018-04-26 03:26:39 +02:00
ThePendulum 5b20a538ff Tracking meta source file for EXIF data. 2018-04-24 22:25:48 +02:00
ThePendulum 8aca700269 Dealing with users without profile. 2018-04-23 14:57:07 +02:00
ThePendulum 4551a1f5d2 Moved source into dedicated directory. 2018-04-23 02:36:07 +02:00