Commit Graph

17 Commits

Author SHA1 Message Date
ThePendulum 66e5bfa50a Added support for RedGifs and Reddit albums. Improved command line logger. Added rate limiters for reddit and host URLs. 2021-12-17 02:04:25 +01:00
ThePendulum e1cc615a99 Added support for various tube and social media sites via youtube-dl. 2019-11-12 03:38:26 +01:00
ThePendulum e175ce38c8 Fetching and storing comments for self posts. 2019-11-05 01:58:00 +01:00
Niels Simenon db6317e560 Removed HTML scraper from imgur module, no longer support due to login restriction. Preserving 'preview' property in index file. 2019-10-31 05:22:07 +01:00
ThePendulum d9ce4dd056 Added option to index (specified) ignored posts. Saving index through save module, and it now notifies the user that an index has been written. 2018-07-07 02:17:41 +02:00
ThePendulum 310931e4f8 Don't attempt to save index when there are no posts or profile details (complained about directory not existing) 2018-07-02 19:55:21 +02:00
ThePendulum 1b1323dc3d No longer redownload profile image and description when indexed, unless --redownload-profile is specified. 2018-07-01 03:26:26 +02:00
ThePendulum e7eef86b23 Added watch-mode. Waiting for profile detail write to finalize before new watch cycle and capture details in index file. 2018-07-01 03:06:57 +02:00
ThePendulum c639503667 Fixed index file for single post fetching. 2018-07-01 01:07:32 +02:00
ThePendulum a859ac573c Accounting for indexed posts. 2018-06-30 21:47:43 +02:00
ThePendulum 74e36a6826 Using YAML rather than TSV for index files. Improves both readability and reindexability. 2018-06-30 03:33:30 +02:00
ThePendulum 33ef3d2c19 Reading index file and ignoring already indexed content. 2018-06-17 03:39:12 +02:00
ThePendulum 7cf1a99915 Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist). 2018-06-17 01:11:10 +02:00
ThePendulum aef4dd02c7 Fetching and curating posts per user without merge. 2018-06-10 02:48:49 +02:00
ThePendulum e056acfbd3 Refactored post retrieval so limit is applied per-user and ignores directly requested posts, and to start utilizing async/await. 2018-05-28 01:42:46 +02:00
ThePendulum edf77e259b Refactored post fetching. Cleaner code that will now fetch archived posts even if the user no longer exists on reddit. Expanded gfycat URL detection. 2018-05-22 22:55:50 +02:00
ThePendulum 804d01d7cf Cleaned up entrypoint. 2018-05-05 02:27:15 +02:00