DebaucheryLibrarian/traxxx
2
1
Fork 1
Code Issues 39 Pull Requests 1 Releases Wiki Activity
2,142 Commits 3 Branches 0 Tags
2e2d8a15ac245d40a7677a69996fce5286b6310b
Commit Graph

873 Commits

Author SHA1 Message Date
DebaucheryLibrarian
26d15c0a6f Updated Jules Jordan scraper for Manuel Ferrara and Girl Girl. 2021-12-05 23:43:23 +01:00
DebaucheryLibrarian
e88554666c Reduced media concurrency to assess effect on memory. Moved qu context removal to deep scrape runner. Updated movie graphql queries. 2021-12-05 02:54:55 +01:00
DebaucheryLibrarian
abd063a578 Removed window map from http module to prevent memory pile-up with update scraping. 2021-12-04 00:32:28 +01:00
DebaucheryLibrarian
1918dd4ad5 Removing query methods from XEmpire release. 2021-12-02 00:45:22 +01:00
DebaucheryLibrarian
61daf5f037 Added debug log for jsdom window closing. 2021-12-01 23:44:25 +01:00
DebaucheryLibrarian
60c151ff6a Reduced deep scrape concurrency to 1. 2021-12-01 23:39:09 +01:00
DebaucheryLibrarian
e41f9fa937 Added waitImmediate to deep scrape, reduced concurrency. 2021-12-01 23:30:10 +01:00
DebaucheryLibrarian
e29cbc9fea Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5. 2021-12-01 17:26:13 +01:00
DebaucheryLibrarian
b93a5715cb Updated Jules Jordan scraper for new update page layout. 2021-12-01 00:00:24 +01:00
DebaucheryLibrarian
87094a9498 Replaced cheerio with qu in Gamma scraper. 2021-11-29 02:49:07 +01:00
DebaucheryLibrarian
f04eb72891 Added upcoming scraper to PurgatoryX. 2021-11-28 01:20:39 +01:00
DebaucheryLibrarian
9d7183ac69 Added PurgatoryX scraper. 2021-11-27 23:55:16 +01:00
DebaucheryLibrarian
20d0d860d3 Fixed MindGeek scraper trying to acquire session from mindgeek.com 2021-11-22 02:51:52 +01:00
DebaucheryLibrarian
6b4aa64d74 Improved MindGeek scraper session check to prevent crash when network session isn't available yet. 2021-11-22 02:44:03 +01:00
DebaucheryLibrarian
e81864ebde Removed verbose http queued output. 2021-11-21 00:47:32 +01:00
DebaucheryLibrarian
0305a22a24 Removed stray console log from media module. 2021-11-21 00:43:35 +01:00
DebaucheryLibrarian
2be3ea9bbb Explicitly adding headers to http module result. 2021-11-21 00:41:12 +01:00
DebaucheryLibrarian
98c103c625 Fixed uuid import. Changed fs rmdir to rm. 2021-11-21 00:31:09 +01:00
DebaucheryLibrarian
f0b7678444 Fixed upsert failing on empty insert array due breaking Knex API change. 2021-11-21 00:19:10 +01:00
DebaucheryLibrarian
26539b74a5 Updated dependencies. Added periodic memory logger. 2021-11-20 23:59:48 +01:00
DebaucheryLibrarian
ccb99e278c Added periodic memory logger. 2021-11-20 23:59:15 +01:00
DebaucheryLibrarian
29b8c5e38e Including unextracted scenes in date determination. 2021-10-28 02:10:30 +02:00
DebaucheryLibrarian
0864154a0e Added unextracted property to keep paginating when extracting scenes. 2021-10-28 01:59:53 +02:00
DebaucheryLibrarian
a22c4d5679 Added beforeNetwork hook, used by MindGeek. Added Filthy Kings to Gamma. 2021-10-27 17:19:23 +02:00
DebaucheryLibrarian
100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian
6c5d4389fe Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages. 2021-10-25 02:06:24 +02:00
DebaucheryLibrarian
0e4fd12d70 Added more loggers to http module. 2021-10-20 01:46:56 +02:00
DebaucheryLibrarian
972b15e72d Added memory profiling. 2021-10-17 23:56:49 +02:00
DebaucheryLibrarian
49f891ba44 Ignoring 1-second scene duration from MindGeek API. 2021-10-17 19:59:05 +02:00
DebaucheryLibrarian
167df35d37 Fixed tags module not dealing with empty releases or tags, added origin tag column to chapter tag table. 2021-10-17 00:28:13 +02:00
DebaucheryLibrarian
b6ad2903f3 Storing original tags. 2021-10-10 00:04:21 +02:00
DebaucheryLibrarian
dbff3e9539 Added Brad Montana. Added teaser link to trailer video. 2021-09-28 20:45:22 +02:00
DebaucheryLibrarian
522584711b Added Teen Mega World scraper. 2021-09-17 03:30:49 +02:00
DebaucheryLibrarian
a316da3083 Retrieving Spizoo trailers. 2021-09-14 00:42:51 +02:00
DebaucheryLibrarian
b164c5dad7 Added Spizoo. 2021-09-13 01:29:39 +02:00
DebaucheryLibrarian
d542889827 Added sections and pagination to stash page. 2021-09-12 00:05:45 +02:00
DebaucheryLibrarian
7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian
69a7e8f13d Fixed PornDoe scraper. 2021-08-26 01:14:54 +02:00
DebaucheryLibrarian
23537e0243 Removed Gamma console logs. 2021-08-24 02:56:59 +02:00
DebaucheryLibrarian
245f69a122 Fixed movie bindings. 2021-08-24 02:54:40 +02:00
DebaucheryLibrarian
bb384029ac Disabled eager searching except for entities. Updating movie search document on store. 2021-08-23 01:54:22 +02:00
DebaucheryLibrarian
6a8c9d89cb Using paginated full text search for movies, combined actor search and fetch to allow combining search with filters. 2021-08-22 22:25:20 +02:00
DebaucheryLibrarian
f00e37490c Finished Cum Louder scraper, updated Vixen scraper. Added tag posters. 2021-08-17 19:25:10 +02:00
DebaucheryLibrarian
b08cb46ae5 Removed stray console log. 2021-08-15 13:21:02 +02:00
DebaucheryLibrarian
aacfd1b29d Updated object-merge-advanced API use. 2021-08-15 13:16:48 +02:00
DebaucheryLibrarian
a848d6991b Added Diabolic and Cum Louder, added content type expect option to media sources to fix Vixen thumbnails. 2021-08-09 10:31:12 +02:00
DebaucheryLibrarian
6847ef690c Added Arch Angel, updated BAM Visions scraper to accomodate Arch Angel (different network, same unidentified CMS). 2021-07-06 00:01:44 +02:00
DebaucheryLibrarian
23b41fc4f3 Fixed Bang scraper. Added Kink affiliate, tag photos. 2021-07-05 00:06:18 +02:00
DebaucheryLibrarian
eb7009832a Added rudimentary affiliate banner setup. Separated login and signup disable. Added various tag photos. 2021-06-28 00:05:24 +02:00
DebaucheryLibrarian
d1480da076 Added effective date column. Changed warning page theme. 2021-06-19 18:09:58 +02:00