Commit Graph

590 Commits

Author SHA1 Message Date
DebaucheryLibrarian
07f0249717 Upgraded pg, knex and postgraphile. Updated nvmrc version to latest node LTS. Fixed Gamma scraper not passing request headers to new http module. 2020-12-14 03:16:47 +01:00
DebaucheryLibrarian
71196688ae Removed console log. 2020-12-05 02:44:39 +01:00
DebaucheryLibrarian
f0bec85ef8 Added Fuck'n'Drive and Jizz On Teens latest layouts to First Anal Quest scraper. 2020-12-05 02:24:31 +01:00
DebaucheryLibrarian
2e0fba3de9 Added First Anal Quest and Double View Casting profile scrapers. 2020-12-04 23:53:20 +01:00
DebaucheryLibrarian
be1821b9eb Changed --inspect to --report to avoid conflict with Node's own debug tools. 2020-12-02 21:26:55 +01:00
DebaucheryLibrarian
2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian
6f19a8a642 Using ageFromBirth for age on scene date. Using infinite depth and colors for inspect. 2020-12-01 23:00:14 +01:00
DebaucheryLibrarian
9a61d2305c Added fixed actor age. Added male profiles to Littlr Caprice Dreams scraper. Added various tag photos. 2020-11-29 03:59:47 +01:00
DebaucheryLibrarian
71c884fe48 Improved Little Caprice Dreams scraper. 2020-11-28 00:46:30 +01:00
DebaucheryLibrarian
9a183c7ffb Added channel matching to Little Caprice Dreams. 2020-11-27 03:23:12 +01:00
DebaucheryLibrarian
60485751e2 Added support for upcoming scenes to Bang scraper. 2020-11-26 22:07:54 +01:00
DebaucheryLibrarian
4e559f63e3 Including all children of included networks, separated included children into dedicated property. 2020-11-26 04:26:52 +01:00
DebaucheryLibrarian
1b407254a7 Increased hard limit in release API. 2020-11-26 04:01:01 +01:00
DebaucheryLibrarian
0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian
6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian
54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian
980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian
711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian
8688c28d0f Replaced queueMethod in media sources for new interval/concurrency options. 2020-11-23 00:38:22 +01:00
DebaucheryLibrarian
ca7c8f0afd Updated place resolve module to use new HTTP module. 2020-11-23 00:30:20 +01:00
DebaucheryLibrarian
0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian
3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian
6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian
081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian
b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian
77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian
d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian
df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian
b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian
8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian
2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian
a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00
DebaucheryLibrarian
8586817963 Fixed Insex html table selector. 2020-11-01 03:25:30 +01:00
DebaucheryLibrarian
c1fffe5cdb Added alternative layout to Insex for updated Topgrl and Sexually Broken sites. 2020-11-01 02:58:21 +01:00
DebaucheryLibrarian
3855d96135 Removed 'null' from scene tile URL if no slug is available. 2020-10-30 17:51:47 +01:00
DebaucheryLibrarian
39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian
c37d4ad01f Filtering invalid actors from releases before storing. 2020-10-29 16:06:20 +01:00
DebaucheryLibrarian
b188bc5744 Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability. 2020-10-29 15:20:59 +01:00
DebaucheryLibrarian
4469376dd2 Using temporary table instead of WHERE IN to stack depth error when finding duplicate actors. 2020-10-28 03:50:52 +01:00
DebaucheryLibrarian
bf9b334b73 Adding scraper config by scraper slug to current 'includes' parameter. 2020-10-28 01:36:13 +01:00
DebaucheryLibrarian
1869877178 Added release covers table to flush routine. 2020-10-27 02:40:30 +01:00
DebaucheryLibrarian
7257776ba8 Mapping avatar to media ID in orphan delete. 2020-10-27 02:34:06 +01:00
DebaucheryLibrarian
1d3ec96e8d Detecting profile avatars in orphaned media flush. 2020-10-27 02:09:52 +01:00
DebaucheryLibrarian
99a7bfeb4c Added traxxx dummy network. 2020-10-26 01:42:38 +01:00
DebaucheryLibrarian
0027be65eb Fixed CzechAV to accomodate teasers instead of photos. Fixed error trying to flush non-existent media file. 2020-10-25 21:43:36 +01:00
DebaucheryLibrarian
57a44d6643 Expanded README. 2020-10-25 01:29:25 +02:00
DebaucheryLibrarian
0bd7fca876 Added orphaned media flush and batch release flush. 2020-10-25 00:52:40 +02:00
DebaucheryLibrarian
6791053c83 Fixed entity alias available through wrong type endpoint. 2020-10-20 21:04:29 +02:00
DebaucheryLibrarian
9f3c686913 Added thumbnail and favicon to entity REST API. 2020-10-20 15:37:42 +02:00
DebaucheryLibrarian
ecdd6d8fb0 Added logo path to entity API. 2020-10-20 15:28:58 +02:00