boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,614 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

464 Commits

Author SHA1 Message Date
DebaucheryLibrarian e3ef0a0d69 Added Top Web Models profile scraper. 2021-01-16 04:10:43 +01:00
DebaucheryLibrarian b9e4764516 Fixed Pascals Sub Sluts interpreting metric as imperial height, filtering unlikely in interpolation. Splitting double actor entries in Top Web Models. 2021-01-15 16:14:48 +01:00
DebaucheryLibrarian b8df8e6507 Added Top Web Models update and scene scraper. 2021-01-15 04:04:32 +01:00
DebaucheryLibrarian 451ffdc48b Added Top Web Models directory. 2021-01-14 02:10:35 +01:00
DebaucheryLibrarian 12c3f0d7b9 Added Top Web Models framework. 2021-01-14 01:37:50 +01:00
DebaucheryLibrarian 39d149c728 Added default deep scrape fetch method. Added Karups scene and profile scraper. Added schoolgirl tag photo. 2021-01-13 21:29:05 +01:00
DebaucheryLibrarian f95c80f73f Using poster photo for Pinky XXX. 2021-01-13 01:10:29 +01:00
DebaucheryLibrarian 52356f0f31 Added Pinky XXX, changed ethnicity tags. 2021-01-13 00:52:05 +01:00
DebaucheryLibrarian ad73c11cb4 Using date and title for Nubiles entry ID. 2021-01-11 23:31:33 +01:00
DebaucheryLibrarian db4e74fb99 Fixed Nubiles base poster query, handling trailing commas in qu source set. Added profile scene scraper to Dogfart. Added tag photo. 2021-01-11 16:20:01 +01:00
DebaucheryLibrarian e38922f372 Removed redundant sitename from MindGeek session error. 2021-01-05 16:35:49 +01:00
DebaucheryLibrarian e1d6c9e489 Added site name to MindGeek session error. 2021-01-05 16:34:32 +01:00
DebaucheryLibrarian 9ca2ec6dd0 Fixed parent entity relations in seed file. Fixed MindGeek scraper session URL determination. 2021-01-05 16:27:20 +01:00
DebaucheryLibrarian 7bbb2f3557 Added tag filter dialog. 2021-01-03 22:53:51 +01:00
DebaucheryLibrarian 8739ec08cf Fixed qu init selector. Fixed Aziani scene page scope. 2021-01-02 03:20:39 +01:00
DebaucheryLibrarian 236d4fcde7 Fixed scene media lazy image. Fixed Perv City actor scope. 2021-01-01 04:28:10 +01:00
DebaucheryLibrarian 361e8f1bd3 Fixed Kelly Madison title regex. 2020-12-29 18:53:55 +01:00
DebaucheryLibrarian bc944c2373 Restored Mike Adriano scraper. 2020-12-29 02:05:22 +01:00
DebaucheryLibrarian 8dd10f7e77 Adjusting tooltip arrow position, added open and close events. Fixed search tooltip layout. 2020-12-29 00:42:02 +01:00
DebaucheryLibrarian 6baa6b0802 Fixed fallback create dates in scene tiles. Fixed Mike Adriano entryIds and trailers for Nympho. 2020-12-28 01:29:34 +01:00
DebaucheryLibrarian 9c926a1d81 Fixed Mike Adriano dates for scene pages. 2020-12-27 23:36:15 +01:00
DebaucheryLibrarian be4d025505 Fixed trailer path regex for some Hush sites. 2020-12-19 02:35:41 +01:00
DebaucheryLibrarian 4ccd8bf07a Fixed missing initial value in scraper reduce, breaking first network (21 Naturals). 2020-12-19 00:19:38 +01:00
DebaucheryLibrarian cd8e810c35 Fixed various Kelly Madison scraper issues. 2020-12-17 02:05:01 +01:00
DebaucheryLibrarian 07f0249717 Upgraded pg, knex and postgraphile. Updated nvmrc version to latest node LTS. Fixed Gamma scraper not passing request headers to new http module. 2020-12-14 03:16:47 +01:00
DebaucheryLibrarian 71196688ae Removed console log. 2020-12-05 02:44:39 +01:00
DebaucheryLibrarian f0bec85ef8 Added Fuck'n'Drive and Jizz On Teens latest layouts to First Anal Quest scraper. 2020-12-05 02:24:31 +01:00
DebaucheryLibrarian 2e0fba3de9 Added First Anal Quest and Double View Casting profile scrapers. 2020-12-04 23:53:20 +01:00
DebaucheryLibrarian be1821b9eb Changed --inspect to --report to avoid conflict with Node's own debug tools. 2020-12-02 21:26:55 +01:00
DebaucheryLibrarian 2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian 9a61d2305c Added fixed actor age. Added male profiles to Littlr Caprice Dreams scraper. Added various tag photos. 2020-11-29 03:59:47 +01:00
DebaucheryLibrarian 71c884fe48 Improved Little Caprice Dreams scraper. 2020-11-28 00:46:30 +01:00
DebaucheryLibrarian 9a183c7ffb Added channel matching to Little Caprice Dreams. 2020-11-27 03:23:12 +01:00
DebaucheryLibrarian 60485751e2 Added support for upcoming scenes to Bang scraper. 2020-11-26 22:07:54 +01:00
DebaucheryLibrarian 4e559f63e3 Including all children of included networks, separated included children into dedicated property. 2020-11-26 04:26:52 +01:00
DebaucheryLibrarian 0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian 6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian 54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian 980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian 711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian 8688c28d0f Replaced queueMethod in media sources for new interval/concurrency options. 2020-11-23 00:38:22 +01:00
DebaucheryLibrarian 0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian 3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian 6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian 081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian 77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian 8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian 2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00
DebaucheryLibrarian 8586817963 Fixed Insex html table selector. 2020-11-01 03:25:30 +01:00
DebaucheryLibrarian c1fffe5cdb Added alternative layout to Insex for updated Topgrl and Sexually Broken sites. 2020-11-01 02:58:21 +01:00
DebaucheryLibrarian 3855d96135 Removed 'null' from scene tile URL if no slug is available. 2020-10-30 17:51:47 +01:00
DebaucheryLibrarian 39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian c37d4ad01f Filtering invalid actors from releases before storing. 2020-10-29 16:06:20 +01:00
DebaucheryLibrarian b188bc5744 Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability. 2020-10-29 15:20:59 +01:00
DebaucheryLibrarian bf9b334b73 Adding scraper config by scraper slug to current 'includes' parameter. 2020-10-28 01:36:13 +01:00
DebaucheryLibrarian 1d3ec96e8d Detecting profile avatars in orphaned media flush. 2020-10-27 02:09:52 +01:00
DebaucheryLibrarian 99a7bfeb4c Added traxxx dummy network. 2020-10-26 01:42:38 +01:00
DebaucheryLibrarian 0027be65eb Fixed CzechAV to accomodate teasers instead of photos. Fixed error trying to flush non-existent media file. 2020-10-25 21:43:36 +01:00
DebaucheryLibrarian 0bd7fca876 Added orphaned media flush and batch release flush. 2020-10-25 00:52:40 +02:00
DebaucheryLibrarian c94e1aaea9 Handling missing trailers in Kink scraper. 2020-09-18 23:25:15 +02:00
DebaucheryLibrarian 3789ef51f2 (Temporarily) removed studio filter from entity query for performance reasons. 2020-09-18 22:43:45 +02:00
DebaucheryLibrarian 3c9468b0f1 Fixed wrong MindGeek session acquire URL. 2020-09-18 03:27:00 +02:00
DebaucheryLibrarian a4929819df Using channel URL instead of composed URL for session retrieval, should fix Brazzers. 2020-09-18 02:54:05 +02:00
DebaucheryLibrarian a9fa71e455 Fixed predata parameter in Assylum scraper. 2020-09-17 14:49:45 +02:00
DebaucheryLibrarian 1a8de4fcf6 Added mimetype verification option to media source to ensure server returned a plausible file. Added additional fallbacks to Jules Jordan poster scraper for Amateur Allure. 2020-09-17 02:30:58 +02:00
DebaucheryLibrarian 0190ee9531 Fixed Porn Doe's poster query. Checking style attribute existence in qu before attempting to use it. 2020-09-17 00:35:41 +02:00
DebaucheryLibrarian 6fb15fb591 Added Dorcel Club with scene and actor scraping. Added count method to qu. 2020-09-16 04:55:30 +02:00
DebaucheryLibrarian a8c525f4fc Hard-coded Pascal White as the male actor for Pascal's Sub Sluts. Added female gender to all Sub Sluts. 2020-09-16 02:38:59 +02:00
DebaucheryLibrarian 7c4dd03a8c Ignoring 'Lockdown Submissions' as actor for Pascals Subsluts. 2020-09-16 01:45:58 +02:00
DebaucheryLibrarian 7fd7005776 Added profile scraper to Pascals Subsluts. 2020-09-16 01:42:15 +02:00
DebaucheryLibrarian bc34c6edb4 Added Bang Bros Vault channel. 2020-09-14 16:01:27 +02:00
DebaucheryLibrarian ba7419d3b0 Added basic Pascals Subsluts scraper. 2020-09-14 02:40:27 +02:00
DebaucheryLibrarian beeaebbfb7 Added CzechAV. 2020-09-14 00:53:41 +02:00
DebaucheryLibrarian eb6337f6fb Fixed upcoming Jules Jordan scene without teaser breaking scraper. 2020-09-11 02:36:36 +02:00
DebaucheryLibrarian 0e8024adf1 Changed next page determination to ensure --after is followed even if there are no unique releases. 2020-09-10 23:49:24 +02:00
DebaucheryLibrarian bec097f14d Returning status codes from Gamma fetch. 2020-09-10 17:54:23 +02:00
DebaucheryLibrarian 7f8704ee2c Using http instead of bhttp for Gamma. 2020-09-10 17:41:29 +02:00
DebaucheryLibrarian 5ee5b270ef Added avatars to Gamma latest API. 2020-09-10 03:56:09 +02:00
DebaucheryLibrarian 88eeab410b Added filter parameter to Gamma scraper to distinguish Evil Angel exclusives from channel scenes. 2020-09-10 03:43:16 +02:00
DebaucheryLibrarian 03ba35d65a Fixed include object. Fixed qu's undefined URL handling. 2020-09-08 03:26:34 +02:00
DebaucheryLibrarian e90bb63a8f Added American Pornstar. Improved Jules Jordan scraper to accomodate for American Pornstar. Changed entity logo mogrify settings to ensure both minimum height and width. 2020-09-05 01:56:54 +02:00
DebaucheryLibrarian 3c84a814a8 Added Zero Tolerance with Addicted 2 Girls and GenderX. 2020-09-04 03:07:28 +02:00
DebaucheryLibrarian c187a27123 Added Hookup Hotshot. 2020-09-03 22:22:12 +02:00
DebaucheryLibrarian f6353ca14c Storing actor profile URL when provided from scene page. 2020-08-31 02:43:41 +02:00
DebaucheryLibrarian 1bfdf4b232 Storing actor profiles from scene pages. 2020-08-30 04:18:47 +02:00
DebaucheryLibrarian 53b0101a12 Added new Naughty America sites. 2020-08-26 23:59:29 +02:00
DebaucheryLibrarian 8611d738b0 Using UTC to query date ranges. Removed stray console log from MindGeek scraper. 2020-08-26 02:01:38 +02:00
DebaucheryLibrarian 52f66e7982 Fixed undefined location in FreeOnes scraper. 2020-08-24 18:24:07 +02:00
DebaucheryLibrarian 7fed5b7138 Moved Brazzers to MindGeek scraper to support new site. 2020-08-24 05:13:34 +02:00
DebaucheryLibrarian c3d771c8fc Hush scraper uses children from entity argument for filter regexp, instead of making its own database request. 2020-08-23 03:31:37 +02:00
DebaucheryLibrarian 278b74e78c Providing duplicate releases in predata. Using duplicates for filtering scenes without channel in Hush scraper. 2020-08-23 02:43:10 +02:00
DebaucheryLibrarian 3a5ea3dd9a Returning duplicate releases from pagination. 2020-08-22 04:22:56 +02:00
DebaucheryLibrarian 5f5c48ea05 Fixed pagination behavior for upcoming scenes. 2020-08-22 01:57:23 +02:00
DebaucheryLibrarian 4ec89e2cc8 Added upcoming, profile and detailed scene actor scraping to InTheCrack. Fixed clip upsert. 2020-08-20 23:35:18 +02:00
DebaucheryLibrarian 501e764c21 Renamed chapters to clips. Fixed Vixen trailers. 2020-08-20 19:52:02 +02:00