boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,583 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

66 Commits

Author SHA1 Message Date
DebaucheryLibrarian 5ed408226d Fixed eye color bruin mapping to bruin instead of brown. 2024-10-18 04:05:51 +02:00
DebaucheryLibrarian 1163b010fb Added actor page deep scrape for the few Sperm Mania scenes not on the homepage. 2024-10-17 23:50:55 +02:00
DebaucheryLibrarian 1950dd2e62 Added Snow Valley (Sperm Mania) scraper. 2024-10-16 02:39:11 +02:00
DebaucheryLibrarian d3a978c501 Using unprint as default, marked to-be-updated scrapers as deprecated. Merging movie tags and movie scene tags for manticore movies table. Removed poster set to null in deep merge; annotate if it has purpose. Refactored Brad Montana scraper. 2024-08-20 02:33:42 +02:00
DebaucheryLibrarian bca677b0a8 Added virtual entity spawning for multi-page updates (i.e. Elegant Angel). Fixed ffmpeg error freezing process. Refactored Adult Empire/Elegant Angel scraper. 2024-08-16 23:26:52 +02:00
DebaucheryLibrarian b8e7029cef Using base poster as fallback for deep poster in merge. Improved Naughty America scraper for live scenes. 2024-07-09 02:19:23 +02:00
DebaucheryLibrarian 88a56794aa Separated Filthy Kings into its channels, upgraded Gamma scraper to accomodate. 2023-07-09 04:35:30 +02:00
DebaucheryLibrarian 0905847ffa Updated Jules Jordan scraper. 2023-07-06 00:14:38 +02:00
DebaucheryLibrarian 205102ff90 Added Meiden van Holland and Vurig Vlaanderen. 2023-07-01 21:46:44 +02:00
DebaucheryLibrarian adda78f0c6 Refactored New Sensations scraper. 2023-06-05 02:13:36 +02:00
DebaucheryLibrarian 164757ee26 Matching URLs to entity using hostname rather than slug to minimize collisions. Fixed missing Cum Louder POV logo. 2023-06-04 21:50:59 +02:00
DebaucheryLibrarian cd98b9d147 Changed deep merge so base release photos are only used when no deep photos are available. 2022-12-29 23:07:08 +01:00
DebaucheryLibrarian 74214bc060 Added puppeteer to http module, refactored Kink scraper to utilize it. 2022-12-28 01:34:12 +01:00
DebaucheryLibrarian 6edd62c337 Refactored Bang! scraper to match new website, first use of unprint. 2022-11-27 04:22:58 +01:00
DebaucheryLibrarian 2943aea4d8 Added showcased migration. Added Love Her Films scraper. 2022-10-25 23:13:24 +02:00
DebaucheryLibrarian 1d84830423 Supporting JSON in scenes file. Added Dogfart link finding utility. 2022-04-07 16:06:38 +02:00
DebaucheryLibrarian 5deba6b90f Passing resolved parameters into before-scrape methods. 2022-02-25 22:13:41 +01:00
DebaucheryLibrarian 3d349c413f Fixed Adult Time scraper. 2022-02-07 22:16:43 +01:00
DebaucheryLibrarian 280032937f Added centralized movie page fetcher. Added memory heap dump. 2021-12-20 02:22:10 +01:00
DebaucheryLibrarian e88554666c Reduced media concurrency to assess effect on memory. Moved qu context removal to deep scrape runner. Updated movie graphql queries. 2021-12-05 02:54:55 +01:00
DebaucheryLibrarian abd063a578 Removed window map from http module to prevent memory pile-up with update scraping. 2021-12-04 00:32:28 +01:00
DebaucheryLibrarian 61daf5f037 Added debug log for jsdom window closing. 2021-12-01 23:44:25 +01:00
DebaucheryLibrarian 60c151ff6a Reduced deep scrape concurrency to 1. 2021-12-01 23:39:09 +01:00
DebaucheryLibrarian e41f9fa937 Added waitImmediate to deep scrape, reduced concurrency. 2021-12-01 23:30:10 +01:00
DebaucheryLibrarian e29cbc9fea Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5. 2021-12-01 17:26:13 +01:00
DebaucheryLibrarian 9d7183ac69 Added PurgatoryX scraper. 2021-11-27 23:55:16 +01:00
DebaucheryLibrarian 26539b74a5 Updated dependencies. Added periodic memory logger. 2021-11-20 23:59:48 +01:00
DebaucheryLibrarian a22c4d5679 Added beforeNetwork hook, used by MindGeek. Added Filthy Kings to Gamma. 2021-10-27 17:19:23 +02:00
DebaucheryLibrarian 100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian 7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian b08cb46ae5 Removed stray console log. 2021-08-15 13:21:02 +02:00
DebaucheryLibrarian aacfd1b29d Updated object-merge-advanced API use. 2021-08-15 13:16:48 +02:00
DebaucheryLibrarian c2a008afbe Added mimetype check to teasers and trailers. Added chapters to MindGeek scraper, fixed scene ID extraction getting stuck on numbers in domain name. Ordering chapters by timestamp. 2021-02-27 18:05:06 +01:00
DebaucheryLibrarian 7ff222ce25 Passing recursive parameters to all scraper methods. Using throttle parameters in MindGeek scraper, fixed missing slug breaking scene and actor URLs. 2021-02-10 03:23:48 +01:00
DebaucheryLibrarian a6c7c60290 Improved movie tile actor overflow. Improved deep movie error feedback. 2021-02-10 03:00:17 +01:00
DebaucheryLibrarian ff123b99b7 Added WankzVR update, scene and profile scraper. 2021-02-04 01:13:02 +01:00
DebaucheryLibrarian 3bbc5a5e87 Hard merging covers to preserve order. 2021-02-02 03:10:58 +01:00
DebaucheryLibrarian d5cdfb36a9 Selecting included networks with infinite parent depth to facilitate scraper resolve. 2021-02-02 01:31:12 +01:00
DebaucheryLibrarian aade7490f8 Querying infinite parent depth for deep release entities. 2021-02-01 01:45:30 +01:00
DebaucheryLibrarian 9535ab7953 Regarding layout parameters from parent and grandparent entities. Removed Evil Angel and Fantasy Massage scraper wrappers. 2021-01-30 00:01:40 +01:00
DebaucheryLibrarian b506a00e7d Improved scene merging. Improved Porn World/DDF scraper for poster and title redundancy. Fixed SFW poster showing in NSFW mode. 2021-01-25 23:53:56 +01:00
DebaucheryLibrarian e9cbf5dab2 Added Woodman Casting X and WUNF update and scene scraper. Fixed actor and fallback media arrays being merged in deep scraper. 2021-01-24 23:31:28 +01:00
DebaucheryLibrarian 39d149c728 Added default deep scrape fetch method. Added Karups scene and profile scraper. Added schoolgirl tag photo. 2021-01-13 21:29:05 +01:00
DebaucheryLibrarian 2bae6f693e Removed stray console log. 2021-01-13 16:08:53 +01:00
DebaucheryLibrarian ae0efccb04 Skipping Babel, updated node version. Improved deep scrape array merge. 2021-01-13 16:08:19 +01:00
DebaucheryLibrarian 77ec2d3747 Using advanced merge library to maximize scraped data. 2020-12-27 23:43:08 +01:00
DebaucheryLibrarian 2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian 980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian 3a5ea3dd9a Returning duplicate releases from pagination. 2020-08-22 04:22:56 +02:00