Commit Graph

19 Commits

Author SHA1 Message Date
DebaucheryLibrarian
711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian
0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian
3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian
b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian
39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian
9ef5ea8fb6 Added the awkward pagination to PornCZ scraper. 2020-07-23 00:55:55 +02:00
DebaucheryLibrarian
46c6c4dd21 Added PornCZ. 2020-07-22 04:12:20 +02:00
749864e922 Added 'independent' property for network-like channels. Changed release tile design. Adding Brazzers timeline events as tags. Added Property Sex to MindGeek. Changed DP, DAP and DVP tag slugs. Changed Porn Pros logo. Added better BAM Visions and Mug Fucked logos. 2020-07-02 04:04:28 +02:00
1f444e58ce Allowing image sources to specify queue method. Using 5s queue for Whale Member to avoid CDN time-outs. 2020-07-01 04:47:05 +02:00
05ee57378a Storing actor avatars. Using 1 second interval queue for location resolve as per OSM code of conduct. 2020-05-16 04:36:45 +02:00
11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
af9d8f7858 Drastically improved memory performance media module using streams and temp files. 2020-05-07 01:53:07 +02:00
2ac879d276 Fixed Vixen scraper, using new token URL for trailers. 2020-04-26 03:51:59 +02:00
9a712e7371 Media module saves files. 2020-03-31 04:05:31 +02:00
4eaacf5697 Expanded new media module. Added network to channel site to fix actor glitch. 2020-03-30 03:01:08 +02:00
fdb2b132f6 Improved release storage module. Added new tags module. Added movie scraping. 2020-03-22 03:50:24 +01:00
d765543b30 Improved update runner. Improved HTTP module API, added default user agent. Added PornCZ and Czechav logos. 2020-03-21 02:48:24 +01:00
4b310e9dfa Added configurable proxy to HTTP module (also used by qu). Added network and site URL to search documents. 2020-03-19 01:54:25 +01:00
349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00