Commit Graph

34 Commits

Author SHA1 Message Date
DebaucheryLibrarian ccb99e278c Added periodic memory logger. 2021-11-20 23:59:15 +01:00
DebaucheryLibrarian f00e37490c Finished Cum Louder scraper, updated Vixen scraper. Added tag posters. 2021-08-17 19:25:10 +02:00
DebaucheryLibrarian a848d6991b Added Diabolic and Cum Louder, added content type expect option to media sources to fix Vixen thumbnails. 2021-08-09 10:31:12 +02:00
DebaucheryLibrarian f8a3bf6a64 Updated scene URLs in Vixen scraper. 2021-04-22 19:49:11 +02:00
DebaucheryLibrarian bb20659934 Refactored clips into chapters. 2021-02-27 00:37:22 +01:00
DebaucheryLibrarian 4b5cd50122 Fixed slug lookup in Perfect Gonzo scraper. 2021-02-01 20:49:08 +01:00
DebaucheryLibrarian 4151412156 Fixed Vixen trailer scraping. Using album instead of expand for actor photos. 2021-01-24 17:16:55 +01:00
DebaucheryLibrarian 0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian 6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian 54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian 3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian 501e764c21 Renamed chapters to clips. Fixed Vixen trailers. 2020-08-20 19:52:02 +02:00
DebaucheryLibrarian dff4d15872 Updated profile scrapers to use base actor instead of actor name. Fixes for Reality Kings and Cherry Pimps scrapers. 2020-07-21 01:44:51 +02:00
DebaucheryLibrarian 6adfded074 Refactored Mike Adriano scraper. Changed logo and favicon. Added style methods to qu. 2020-07-16 03:47:07 +02:00
ThePendulum 885aa4f627 Passing context object with site or network instead of scraper slug and 'site or network' to all profile scrapers. 2020-05-18 03:22:03 +02:00
ThePendulum 11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
ThePendulum a223f933ce Added 2 minute timeout to media fetch. 2020-04-26 04:21:57 +02:00
ThePendulum 2ac879d276 Fixed Vixen scraper, using new token URL for trailers. 2020-04-26 03:51:59 +02:00
ThePendulum 01b59f0a9b Changed withReleases boolean to include object in Vixen and Fame Digital scrapers. 2020-03-12 02:21:59 +01:00
ThePendulum 6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
ThePendulum 915eb75719 Refactored Vixen scraper, using API endpoint and added actor profile and releases scraper. Release scraper will return base release when present and 'deep' argument is false. 2020-02-22 23:25:10 +01:00
ThePendulum 5e1a1005f1 Refactored Vixen scene scraper. Using better poster source for Vixen. Returning video as teaser instead of trailer. 2020-02-22 05:29:02 +01:00
ThePendulum e5c6ccd252 Scraping upcoming Vixen scenes. Fetching release media groups sequentially to prevent collisions. 2020-02-22 04:37:48 +01:00
ThePendulum 32a0188b72 Fixed Boobpedia scraper. Catching non-OK responses for Vixen scraper. 2020-01-27 03:07:06 +01:00
ThePendulum 4e4323704a Getting more and higher quality Vixen photos. 2020-01-18 23:10:31 +01:00
ThePendulum b489c8fc33 Major refactor, cleand up site scrape module, fixed and cleaned up release scrape module. Removed old CLI code 2019-11-16 03:33:36 +01:00
ThePendulum 681f1bca85 Added tag page. Added default 'anal' tag to Vixen scraper for Tushy and Tushy Raw. 2019-11-11 05:18:28 +01:00
ThePendulum 298eabe56e Improved Vixen and XEmpire scrapers. Added media to Blowpass scraper. Improved release fetch code. 2019-09-26 03:27:01 +02:00
ThePendulum 96d8cfbcb6 Improved media handling, added trailer support. Fetching media from Vixen network frontpages. 2019-09-25 04:52:58 +02:00
ThePendulum b12d74cbb3 Fixed Vixen scraper, site restructured. Added Deeper to Vixen network. 2019-09-25 02:44:25 +02:00
ThePendulum 6b9d99f495 Added description property to recent scrapers (apparently forgotten). Reconfigured eslint for scrapers. 2019-04-07 05:19:44 +02:00
ThePendulum 2b818e379a Scrapers can now iterate through pages. Filtering unique releases before saving to database. Improved scrapers and rendering. 2019-04-05 03:45:40 +02:00
ThePendulum b3beeef3e4 Added Private scraper. Added Vixen scraper to repository. 2019-04-04 04:00:28 +02:00