boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,881 Commits 7 Branches 0 Tags 3.5 GiB
Commit Graph

771 Commits

Author SHA1 Message Date
DebaucheryLibrarian e88554666c Reduced media concurrency to assess effect on memory. Moved qu context removal to deep scrape runner. Updated movie graphql queries. 2021-12-05 02:54:55 +01:00
DebaucheryLibrarian abd063a578 Removed window map from http module to prevent memory pile-up with update scraping. 2021-12-04 00:32:28 +01:00
DebaucheryLibrarian 1918dd4ad5 Removing query methods from XEmpire release. 2021-12-02 00:45:22 +01:00
DebaucheryLibrarian 61daf5f037 Added debug log for jsdom window closing. 2021-12-01 23:44:25 +01:00
DebaucheryLibrarian 60c151ff6a Reduced deep scrape concurrency to 1. 2021-12-01 23:39:09 +01:00
DebaucheryLibrarian e41f9fa937 Added waitImmediate to deep scrape, reduced concurrency. 2021-12-01 23:30:10 +01:00
DebaucheryLibrarian e29cbc9fea Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5. 2021-12-01 17:26:13 +01:00
DebaucheryLibrarian b93a5715cb Updated Jules Jordan scraper for new update page layout. 2021-12-01 00:00:24 +01:00
DebaucheryLibrarian 87094a9498 Replaced cheerio with qu in Gamma scraper. 2021-11-29 02:49:07 +01:00
DebaucheryLibrarian f04eb72891 Added upcoming scraper to PurgatoryX. 2021-11-28 01:20:39 +01:00
DebaucheryLibrarian 9d7183ac69 Added PurgatoryX scraper. 2021-11-27 23:55:16 +01:00
DebaucheryLibrarian 20d0d860d3 Fixed MindGeek scraper trying to acquire session from mindgeek.com 2021-11-22 02:51:52 +01:00
DebaucheryLibrarian 6b4aa64d74 Improved MindGeek scraper session check to prevent crash when network session isn't available yet. 2021-11-22 02:44:03 +01:00
DebaucheryLibrarian e81864ebde Removed verbose http queued output. 2021-11-21 00:47:32 +01:00
DebaucheryLibrarian 0305a22a24 Removed stray console log from media module. 2021-11-21 00:43:35 +01:00
DebaucheryLibrarian 2be3ea9bbb Explicitly adding headers to http module result. 2021-11-21 00:41:12 +01:00
DebaucheryLibrarian 98c103c625 Fixed uuid import. Changed fs rmdir to rm. 2021-11-21 00:31:09 +01:00
DebaucheryLibrarian f0b7678444 Fixed upsert failing on empty insert array due breaking Knex API change. 2021-11-21 00:19:10 +01:00
DebaucheryLibrarian 26539b74a5 Updated dependencies. Added periodic memory logger. 2021-11-20 23:59:48 +01:00
DebaucheryLibrarian 29b8c5e38e Including unextracted scenes in date determination. 2021-10-28 02:10:30 +02:00
DebaucheryLibrarian 0864154a0e Added unextracted property to keep paginating when extracting scenes. 2021-10-28 01:59:53 +02:00
DebaucheryLibrarian a22c4d5679 Added beforeNetwork hook, used by MindGeek. Added Filthy Kings to Gamma. 2021-10-27 17:19:23 +02:00
DebaucheryLibrarian 100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian 6c5d4389fe Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages. 2021-10-25 02:06:24 +02:00
DebaucheryLibrarian 0e4fd12d70 Added more loggers to http module. 2021-10-20 01:46:56 +02:00
DebaucheryLibrarian 972b15e72d Added memory profiling. 2021-10-17 23:56:49 +02:00
DebaucheryLibrarian 49f891ba44 Ignoring 1-second scene duration from MindGeek API. 2021-10-17 19:59:05 +02:00
DebaucheryLibrarian 167df35d37 Fixed tags module not dealing with empty releases or tags, added origin tag column to chapter tag table. 2021-10-17 00:28:13 +02:00
DebaucheryLibrarian b6ad2903f3 Storing original tags. 2021-10-10 00:04:21 +02:00
DebaucheryLibrarian dbff3e9539 Added Brad Montana. Added teaser link to trailer video. 2021-09-28 20:45:22 +02:00
DebaucheryLibrarian 522584711b Added Teen Mega World scraper. 2021-09-17 03:30:49 +02:00
DebaucheryLibrarian a316da3083 Retrieving Spizoo trailers. 2021-09-14 00:42:51 +02:00
DebaucheryLibrarian b164c5dad7 Added Spizoo. 2021-09-13 01:29:39 +02:00
DebaucheryLibrarian d542889827 Added sections and pagination to stash page. 2021-09-12 00:05:45 +02:00
DebaucheryLibrarian 7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian 69a7e8f13d Fixed PornDoe scraper. 2021-08-26 01:14:54 +02:00
DebaucheryLibrarian 23537e0243 Removed Gamma console logs. 2021-08-24 02:56:59 +02:00
DebaucheryLibrarian 245f69a122 Fixed movie bindings. 2021-08-24 02:54:40 +02:00
DebaucheryLibrarian bb384029ac Disabled eager searching except for entities. Updating movie search document on store. 2021-08-23 01:54:22 +02:00
DebaucheryLibrarian 6a8c9d89cb Using paginated full text search for movies, combined actor search and fetch to allow combining search with filters. 2021-08-22 22:25:20 +02:00
DebaucheryLibrarian f00e37490c Finished Cum Louder scraper, updated Vixen scraper. Added tag posters. 2021-08-17 19:25:10 +02:00
DebaucheryLibrarian b08cb46ae5 Removed stray console log. 2021-08-15 13:21:02 +02:00
DebaucheryLibrarian aacfd1b29d Updated object-merge-advanced API use. 2021-08-15 13:16:48 +02:00
DebaucheryLibrarian a848d6991b Added Diabolic and Cum Louder, added content type expect option to media sources to fix Vixen thumbnails. 2021-08-09 10:31:12 +02:00
DebaucheryLibrarian 6847ef690c Added Arch Angel, updated BAM Visions scraper to accomodate Arch Angel (different network, same unidentified CMS). 2021-07-06 00:01:44 +02:00
DebaucheryLibrarian 23b41fc4f3 Fixed Bang scraper. Added Kink affiliate, tag photos. 2021-07-05 00:06:18 +02:00
DebaucheryLibrarian eb7009832a Added rudimentary affiliate banner setup. Separated login and signup disable. Added various tag photos. 2021-06-28 00:05:24 +02:00
DebaucheryLibrarian d1480da076 Added effective date column. Changed warning page theme. 2021-06-19 18:09:58 +02:00
DebaucheryLibrarian ab1329dd67 Updating entity ID for rescraped scenes with network entry IDs enabled. 2021-06-13 16:49:27 +02:00
DebaucheryLibrarian e9a0700742 Improved upcoming update query. 2021-06-06 01:09:32 +02:00