boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,901 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

782 Commits

Author SHA1 Message Date
DebaucheryLibrarian 299f257a01 Logging memory usage on media fetch. 2021-12-27 17:29:23 +01:00
DebaucheryLibrarian 3bfe1e8fdb Storing scenes before fetching movies. 2021-12-20 03:29:14 +01:00
DebaucheryLibrarian 7d992f4129 Added triggers to memory dump. 2021-12-20 02:28:59 +01:00
DebaucheryLibrarian 280032937f Added centralized movie page fetcher. Added memory heap dump. 2021-12-20 02:22:10 +01:00
DebaucheryLibrarian dc187a9a3a Added execute method to qu, removed runScripts from Gamma's fetchMovie to observe effect on memory usage. 2021-12-19 23:03:44 +01:00
DebaucheryLibrarian 1c0c30be11 Improved and re-enabled why-is-node-running logger. 2021-12-18 23:44:51 +01:00
DebaucheryLibrarian 8e4be06675 Closing JSDOM window after deep scrapes in an attempt to save memory. 2021-12-18 23:22:35 +01:00
DebaucheryLibrarian 65e2b72c6a Fixed Gamma movie scraper failing when data object is not available. 2021-12-13 01:20:02 +01:00
DebaucheryLibrarian 5b5688438c Fixed Gamma movie scene query. Added tag photos. 2021-12-11 22:46:55 +01:00
DebaucheryLibrarian ec4e7bc12a Updated Jules Jordan scraper for Sperm Swallowers and The Ass Factory. 2021-12-05 23:59:17 +01:00
DebaucheryLibrarian 26d15c0a6f Updated Jules Jordan scraper for Manuel Ferrara and Girl Girl. 2021-12-05 23:43:23 +01:00
DebaucheryLibrarian e88554666c Reduced media concurrency to assess effect on memory. Moved qu context removal to deep scrape runner. Updated movie graphql queries. 2021-12-05 02:54:55 +01:00
DebaucheryLibrarian abd063a578 Removed window map from http module to prevent memory pile-up with update scraping. 2021-12-04 00:32:28 +01:00
DebaucheryLibrarian 1918dd4ad5 Removing query methods from XEmpire release. 2021-12-02 00:45:22 +01:00
DebaucheryLibrarian 61daf5f037 Added debug log for jsdom window closing. 2021-12-01 23:44:25 +01:00
DebaucheryLibrarian 60c151ff6a Reduced deep scrape concurrency to 1. 2021-12-01 23:39:09 +01:00
DebaucheryLibrarian e41f9fa937 Added waitImmediate to deep scrape, reduced concurrency. 2021-12-01 23:30:10 +01:00
DebaucheryLibrarian e29cbc9fea Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5. 2021-12-01 17:26:13 +01:00
DebaucheryLibrarian b93a5715cb Updated Jules Jordan scraper for new update page layout. 2021-12-01 00:00:24 +01:00
DebaucheryLibrarian 87094a9498 Replaced cheerio with qu in Gamma scraper. 2021-11-29 02:49:07 +01:00
DebaucheryLibrarian f04eb72891 Added upcoming scraper to PurgatoryX. 2021-11-28 01:20:39 +01:00
DebaucheryLibrarian 9d7183ac69 Added PurgatoryX scraper. 2021-11-27 23:55:16 +01:00
DebaucheryLibrarian 20d0d860d3 Fixed MindGeek scraper trying to acquire session from mindgeek.com 2021-11-22 02:51:52 +01:00
DebaucheryLibrarian 6b4aa64d74 Improved MindGeek scraper session check to prevent crash when network session isn't available yet. 2021-11-22 02:44:03 +01:00
DebaucheryLibrarian e81864ebde Removed verbose http queued output. 2021-11-21 00:47:32 +01:00
DebaucheryLibrarian 0305a22a24 Removed stray console log from media module. 2021-11-21 00:43:35 +01:00
DebaucheryLibrarian 2be3ea9bbb Explicitly adding headers to http module result. 2021-11-21 00:41:12 +01:00
DebaucheryLibrarian 98c103c625 Fixed uuid import. Changed fs rmdir to rm. 2021-11-21 00:31:09 +01:00
DebaucheryLibrarian f0b7678444 Fixed upsert failing on empty insert array due breaking Knex API change. 2021-11-21 00:19:10 +01:00
DebaucheryLibrarian 26539b74a5 Updated dependencies. Added periodic memory logger. 2021-11-20 23:59:48 +01:00
DebaucheryLibrarian 29b8c5e38e Including unextracted scenes in date determination. 2021-10-28 02:10:30 +02:00
DebaucheryLibrarian 0864154a0e Added unextracted property to keep paginating when extracting scenes. 2021-10-28 01:59:53 +02:00
DebaucheryLibrarian a22c4d5679 Added beforeNetwork hook, used by MindGeek. Added Filthy Kings to Gamma. 2021-10-27 17:19:23 +02:00
DebaucheryLibrarian 100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian 6c5d4389fe Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages. 2021-10-25 02:06:24 +02:00
DebaucheryLibrarian 0e4fd12d70 Added more loggers to http module. 2021-10-20 01:46:56 +02:00
DebaucheryLibrarian 972b15e72d Added memory profiling. 2021-10-17 23:56:49 +02:00
DebaucheryLibrarian 49f891ba44 Ignoring 1-second scene duration from MindGeek API. 2021-10-17 19:59:05 +02:00
DebaucheryLibrarian 167df35d37 Fixed tags module not dealing with empty releases or tags, added origin tag column to chapter tag table. 2021-10-17 00:28:13 +02:00
DebaucheryLibrarian b6ad2903f3 Storing original tags. 2021-10-10 00:04:21 +02:00
DebaucheryLibrarian dbff3e9539 Added Brad Montana. Added teaser link to trailer video. 2021-09-28 20:45:22 +02:00
DebaucheryLibrarian 522584711b Added Teen Mega World scraper. 2021-09-17 03:30:49 +02:00
DebaucheryLibrarian a316da3083 Retrieving Spizoo trailers. 2021-09-14 00:42:51 +02:00
DebaucheryLibrarian b164c5dad7 Added Spizoo. 2021-09-13 01:29:39 +02:00
DebaucheryLibrarian d542889827 Added sections and pagination to stash page. 2021-09-12 00:05:45 +02:00
DebaucheryLibrarian 7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian 69a7e8f13d Fixed PornDoe scraper. 2021-08-26 01:14:54 +02:00
DebaucheryLibrarian 23537e0243 Removed Gamma console logs. 2021-08-24 02:56:59 +02:00
DebaucheryLibrarian 245f69a122 Fixed movie bindings. 2021-08-24 02:54:40 +02:00
DebaucheryLibrarian bb384029ac Disabled eager searching except for entities. Updating movie search document on store. 2021-08-23 01:54:22 +02:00