Commit Graph

3069 Commits

Author SHA1 Message Date
DebaucheryLibrarian
0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian
3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian
6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian
081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian
b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian
5d0fe44130 1.141.2 2020-11-19 02:01:24 +01:00
DebaucheryLibrarian
77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian
ecc90be12c 1.141.1 2020-11-15 23:50:11 +01:00
DebaucheryLibrarian
d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian
b952b758d7 1.141.0 2020-11-15 04:33:40 +01:00
DebaucheryLibrarian
df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian
b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian
cc7c9f3b31 1.140.2 2020-11-13 01:16:27 +01:00
DebaucheryLibrarian
8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian
2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian
e698146d94 1.140.1 2020-11-05 02:22:23 +01:00
DebaucheryLibrarian
4408507371 Added creampie tag photo. 2020-11-05 02:22:15 +01:00
DebaucheryLibrarian
574f53c66d 1.140.0 2020-11-04 04:03:02 +01:00
DebaucheryLibrarian
a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00
DebaucheryLibrarian
ec7acd46a0 1.139.7 2020-11-03 03:36:43 +01:00
DebaucheryLibrarian
6976eb337d Added various tag photos. 2020-11-03 03:36:34 +01:00
DebaucheryLibrarian
994413c509 1.139.6 2020-11-02 04:55:16 +01:00
DebaucheryLibrarian
0078dba085 Added various tag photos. 2020-11-02 04:49:16 +01:00
DebaucheryLibrarian
9a636b81a6 1.139.5 2020-11-01 05:25:38 +01:00
DebaucheryLibrarian
734de26559 Added various tag photos. 2020-11-01 05:25:29 +01:00
DebaucheryLibrarian
aa5e9a9c8b 1.139.4 2020-11-01 03:25:37 +01:00
DebaucheryLibrarian
8586817963 Fixed Insex html table selector. 2020-11-01 03:25:30 +01:00
DebaucheryLibrarian
ccca6a7714 1.139.3 2020-11-01 02:58:29 +01:00
DebaucheryLibrarian
c1fffe5cdb Added alternative layout to Insex for updated Topgrl and Sexually Broken sites. 2020-11-01 02:58:21 +01:00
DebaucheryLibrarian
7c11b2204e 1.139.2 2020-10-31 04:53:50 +01:00
DebaucheryLibrarian
72859ede85 Added various tag photos. 2020-10-31 04:53:40 +01:00
DebaucheryLibrarian
30c002d0f2 1.139.1 2020-10-30 17:51:56 +01:00
DebaucheryLibrarian
3855d96135 Removed 'null' from scene tile URL if no slug is available. 2020-10-30 17:51:47 +01:00
DebaucheryLibrarian
7974be05e9 1.139.0 2020-10-30 17:37:19 +01:00
DebaucheryLibrarian
39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian
4af7597441 1.138.9 2020-10-29 16:06:27 +01:00
DebaucheryLibrarian
c37d4ad01f Filtering invalid actors from releases before storing. 2020-10-29 16:06:20 +01:00
DebaucheryLibrarian
2801732f57 1.138.8 2020-10-29 15:21:08 +01:00
DebaucheryLibrarian
b188bc5744 Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability. 2020-10-29 15:20:59 +01:00
DebaucheryLibrarian
f4b1fb4831 1.138.7 2020-10-28 15:28:43 +01:00
DebaucheryLibrarian
8c553d5b3d Added traxxx dummy network to default excludes. 2020-10-28 15:28:39 +01:00
DebaucheryLibrarian
e40d7ba181 1.138.6 2020-10-28 03:51:15 +01:00
DebaucheryLibrarian
4469376dd2 Using temporary table instead of WHERE IN to stack depth error when finding duplicate actors. 2020-10-28 03:50:52 +01:00
DebaucheryLibrarian
64a52fbb1e 1.138.5 2020-10-28 01:36:21 +01:00
DebaucheryLibrarian
bf9b334b73 Adding scraper config by scraper slug to current 'includes' parameter. 2020-10-28 01:36:13 +01:00
DebaucheryLibrarian
1869877178 Added release covers table to flush routine. 2020-10-27 02:40:30 +01:00
DebaucheryLibrarian
209fe67bb0 1.138.4 2020-10-27 02:34:15 +01:00
DebaucheryLibrarian
7257776ba8 Mapping avatar to media ID in orphan delete. 2020-10-27 02:34:06 +01:00
DebaucheryLibrarian
af7fa56e02 1.138.3 2020-10-27 02:10:03 +01:00
DebaucheryLibrarian
1d3ec96e8d Detecting profile avatars in orphaned media flush. 2020-10-27 02:09:52 +01:00