 ThePendulum
|
74e36a6826
|
Using YAML rather than TSV for index files. Improves both readability and reindexability.
|
2018-06-30 03:33:30 +02:00 |
|
ThePendulum
|
1c4ec06f68
|
Extracting indexed posts from fetched posts to allow for index rewrite flexibility.
|
2018-06-20 02:06:59 +02:00 |
|
ThePendulum
|
33ef3d2c19
|
Reading index file and ignoring already indexed content.
|
2018-06-17 03:39:12 +02:00 |
|
ThePendulum
|
7cf1a99915
|
Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist).
|
2018-06-17 01:11:10 +02:00 |
|
ThePendulum
|
3b5d886da3
|
Saving index per user as TSV. Refactoring.
|
2018-06-15 01:06:31 +02:00 |
|
ThePendulum
|
7446dc9efb
|
Refactored info content fetching and saving to handle users object and utilize async/await.
|
2018-06-13 01:51:45 +02:00 |
|
ThePendulum
|
aef4dd02c7
|
Fetching and curating posts per user without merge.
|
2018-06-10 02:48:49 +02:00 |
|
ThePendulum
|
f579bb2077
|
Remove double slashes from URLs before matching.
|
2018-06-10 01:34:28 +02:00 |
|
ThePendulum
|
3a724d1b37
|
Improved handling of unavailable/deleted vidble videos
|
2018-06-07 14:55:16 +02:00 |
|
ThePendulum
|
504de6247f
|
Added vidble direct video (/watch?v=) support.
|
2018-06-07 14:40:15 +02:00 |
|
ThePendulum
|
89542bd064
|
Added -size-restricted URL pattern for gfycat. Fixed missing getUser argument.
|
2018-05-28 22:25:58 +02:00 |
|
ThePendulum
|
e056acfbd3
|
Refactored post retrieval so limit is applied per-user and ignores directly requested posts, and to start utilizing async/await.
|
2018-05-28 01:42:46 +02:00 |
|
ThePendulum
|
569213e291
|
Added filepath component length limit with truncator. Fixed double extension dot when copying original extension. Moved API keys to local config.
|
2018-05-23 17:03:12 +02:00 |
|
ThePendulum
|
edf77e259b
|
Refactored post fetching. Cleaner code that will now fetch archived posts even if the user no longer exists on reddit. Expanded gfycat URL detection.
|
2018-05-22 22:55:50 +02:00 |
|
ThePendulum
|
67378d4c7a
|
Added new eroshare method source files.
|
2018-05-15 15:36:15 +02:00 |
|
ThePendulum
|
6f21d41cd0
|
Added support for /i/ eroshare items.
|
2018-05-14 00:38:49 +02:00 |
|
ThePendulum
|
b9a9fb16cf
|
Removed jpeg to jpg replace.
|
2018-05-12 23:37:24 +02:00 |
|
ThePendulum
|
8abda731dd
|
Added vidble image support.
|
2018-05-12 23:36:20 +02:00 |
|
ThePendulum
|
9d6882a7c5
|
Fixed jpeg jpg substitution.
|
2018-05-12 16:29:41 +02:00 |
|
ThePendulum
|
adbbc79b16
|
Added support for Vidble (image) albums. Replaced extensions.json and reverseLookup with mime-types lib.
|
2018-05-11 01:46:08 +02:00 |
|
ThePendulum
|
cdff8354a7
|
Fixed missing return causing no saves.
|
2018-05-05 22:39:21 +02:00 |
|
ThePendulum
|
068ffbdbd8
|
Added include and exclude source arguments. Improved stream fetch failure handling and improved logging details.
|
2018-05-05 17:08:40 +02:00 |
|
ThePendulum
|
3794fc3776
|
Added support for m.imgur.com links. Changed curate ignore log message details.
|
2018-05-05 03:18:05 +02:00 |
|
ThePendulum
|
804d01d7cf
|
Cleaned up entrypoint.
|
2018-05-05 02:27:15 +02:00 |
|
ThePendulum
|
9d32450864
|
Added support for reddit preview fallback.
|
2018-05-05 02:14:48 +02:00 |
|
ThePendulum
|
dd3be88122
|
Added preview property. Removed debugging intercepts. Logging post permalink on failed info fetch.
|
2018-05-05 01:22:23 +02:00 |
|
ThePendulum
|
ca3bdd717d
|
Added archive support, and the IP archive.
|
2018-05-05 00:51:58 +02:00 |
|
ThePendulum
|
5b27099028
|
Added support for redd.it PNG.
|
2018-05-01 02:56:56 +02:00 |
|
ThePendulum
|
93781b596f
|
Allow posts from [deleted] users to be fetched.
|
2018-05-01 02:46:13 +02:00 |
|
ThePendulum
|
632dc96bc1
|
Fixed profile description not saving when user has avatar.
|
2018-04-30 14:17:44 +02:00 |
|
ThePendulum
|
4754e2445d
|
Added npm script for running application.
|
2018-04-29 04:13:01 +02:00 |
|
ThePendulum
|
05b5bee7ea
|
Added support for fetching individual posts. Improved use of yargs, --help now available. Refactored main app flow.
|
2018-04-29 02:02:34 +02:00 |
|
ThePendulum
|
45182c07db
|
Checking result status rather than ok for imgur methods.
|
2018-04-28 17:07:16 +02:00 |
|
ThePendulum
|
ce6b63562a
|
Fixed obsolete eroshare extract single album item property. Throwing error when imgur album and image info fetch fails. Main catch method console log colored red.
|
2018-04-28 02:10:55 +02:00 |
|
ThePendulum
|
9e9cca8834
|
Fixed avatar property.
|
2018-04-28 00:27:29 +02:00 |
|
ThePendulum
|
33bc13ebfb
|
Added ignore argument.
|
2018-04-28 00:03:31 +02:00 |
|
ThePendulum
|
ccfdc16ca8
|
Added reddit avatar URL detection and avoid option.
|
2018-04-27 23:14:48 +02:00 |
|
ThePendulum
|
88425aee5b
|
Fixed double dir path component extraction in save module (causing profile info save to fail).
|
2018-04-27 00:42:11 +02:00 |
|
ThePendulum
|
9e9ed9de07
|
Fixed text posts not passing streams correctly since previous change.
|
2018-04-26 04:53:56 +02:00 |
|
ThePendulum
|
0313b291da
|
Adapted primary save function to handle single streams and non-item saves (e.g. profiles).
|
2018-04-26 03:35:42 +02:00 |
|
ThePendulum
|
e752667be7
|
Added support for muxing streams, specifically for reddit-hosted videos (now also supported).
|
2018-04-26 03:26:39 +02:00 |
|
ThePendulum
|
f2052004df
|
Using metadata configuration rather than item description for EXIF save. Fixed dissect duplicate detection failing for empty host.
|
2018-04-26 00:34:08 +02:00 |
|
ThePendulum
|
0b9fba7af2
|
Ignoring empty interpolated metadata. Added URL query wildcard to imgur and reddit images. Using host ID rather than URL to improve duplicate detection.
|
2018-04-26 00:22:56 +02:00 |
|
ThePendulum
|
5b20a538ff
|
Tracking meta source file for EXIF data.
|
2018-04-24 22:25:48 +02:00 |
|
ThePendulum
|
f4535e443b
|
Saving EXIF data for jpegs.
|
2018-04-24 22:25:36 +02:00 |
|
ThePendulum
|
dece4b2530
|
Added support for reddituploads.com. Changed default sorting to 'new'.
|
2018-04-24 00:55:58 +02:00 |
|
ThePendulum
|
8bec57e906
|
Changed ignoreDuplicates to avoidDuplicates.
|
2018-04-23 14:58:23 +02:00 |
|
ThePendulum
|
8aca700269
|
Dealing with users without profile.
|
2018-04-23 14:57:07 +02:00 |
|
ThePendulum
|
b34b73b464
|
Added subreddit variable.
|
2018-04-23 02:44:43 +02:00 |
|
ThePendulum
|
4551a1f5d2
|
Moved source into dedicated directory.
|
2018-04-23 02:36:07 +02:00 |