Commit Graph

485 Commits

Author SHA1 Message Date
DebaucheryLibrarian
100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian
6c5d4389fe Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages. 2021-10-25 02:06:24 +02:00
DebaucheryLibrarian
49f891ba44 Ignoring 1-second scene duration from MindGeek API. 2021-10-17 19:59:05 +02:00
DebaucheryLibrarian
dbff3e9539 Added Brad Montana. Added teaser link to trailer video. 2021-09-28 20:45:22 +02:00
DebaucheryLibrarian
522584711b Added Teen Mega World scraper. 2021-09-17 03:30:49 +02:00
DebaucheryLibrarian
a316da3083 Retrieving Spizoo trailers. 2021-09-14 00:42:51 +02:00
DebaucheryLibrarian
b164c5dad7 Added Spizoo. 2021-09-13 01:29:39 +02:00
DebaucheryLibrarian
7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian
69a7e8f13d Fixed PornDoe scraper. 2021-08-26 01:14:54 +02:00
DebaucheryLibrarian
23537e0243 Removed Gamma console logs. 2021-08-24 02:56:59 +02:00
DebaucheryLibrarian
f00e37490c Finished Cum Louder scraper, updated Vixen scraper. Added tag posters. 2021-08-17 19:25:10 +02:00
DebaucheryLibrarian
a848d6991b Added Diabolic and Cum Louder, added content type expect option to media sources to fix Vixen thumbnails. 2021-08-09 10:31:12 +02:00
DebaucheryLibrarian
6847ef690c Added Arch Angel, updated BAM Visions scraper to accomodate Arch Angel (different network, same unidentified CMS). 2021-07-06 00:01:44 +02:00
DebaucheryLibrarian
23b41fc4f3 Fixed Bang scraper. Added Kink affiliate, tag photos. 2021-07-05 00:06:18 +02:00
DebaucheryLibrarian
ab1329dd67 Updating entity ID for rescraped scenes with network entry IDs enabled. 2021-06-13 16:49:27 +02:00
DebaucheryLibrarian
c979173422 Rescraping upcoming scenes. Fixed language and scene deep scraping for Dorcel scraper. 2021-06-02 03:27:32 +02:00
DebaucheryLibrarian
f8a3bf6a64 Updated scene URLs in Vixen scraper. 2021-04-22 19:49:11 +02:00
DebaucheryLibrarian
95f3b1c03a Added rudimentary notifications for set alerts. 2021-04-17 01:10:45 +02:00
DebaucheryLibrarian
010da8954b Fixed MYLF scraper failing when channel is missing, fixed profile measurement matching. Added MYLF Selects channel. 2021-03-29 22:47:43 +02:00
DebaucheryLibrarian
c386a9098f Generalized Team Skeet scraper, added MYLF network and various Team Skeet partner channels. 2021-03-29 22:22:56 +02:00
DebaucheryLibrarian
193af9bab5 Fixed session options in http module. 2021-03-23 15:25:21 +01:00
DebaucheryLibrarian
c2a008afbe Added mimetype check to teasers and trailers. Added chapters to MindGeek scraper, fixed scene ID extraction getting stuck on numbers in domain name. Ordering chapters by timestamp. 2021-02-27 18:05:06 +01:00
DebaucheryLibrarian
bb20659934 Refactored clips into chapters. 2021-02-27 00:37:22 +01:00
DebaucheryLibrarian
20129eca5d Fixed Bang scraper. 2021-02-26 02:52:39 +01:00
DebaucheryLibrarian
2b5aac7633 Fixed S3 display support for movies. 2021-02-23 01:30:38 +01:00
DebaucheryLibrarian
37e39dc1ec Added S3 support for media files. Fixed MindGeek scraper for new poster data structure. 2021-02-22 02:33:39 +01:00
DebaucheryLibrarian
b79e75349c Fixed indentation. 2021-02-21 22:58:46 +01:00
DebaucheryLibrarian
8a739893ea Improved duplicate handling. Added tag photos. 2021-02-21 22:17:25 +01:00
DebaucheryLibrarian
a39750d808 Fixed Nubiles deep scrape date. 2021-02-20 01:16:09 +01:00
DebaucheryLibrarian
c51cd080fa Improved actor mapping in release associations. Storing alias ID in actor release association. 2021-02-17 00:40:20 +01:00
DebaucheryLibrarian
3469da674a Fixed PornCZ video query. 2021-02-16 19:53:32 +01:00
DebaucheryLibrarian
67055bf920 Improved actor entity and entry ID storage. 2021-02-16 03:37:52 +01:00
DebaucheryLibrarian
8a7baa02c1 Improved date query for upcoming scenes in ElevatedX scraper. 2021-02-14 14:31:33 +01:00
DebaucheryLibrarian
372b5da704 Moved all of ExploitedX and Nebraska Coeds into generic ElevatedX scraper. 2021-02-14 01:53:03 +01:00
DebaucheryLibrarian
f79505f3f6 Added generic ElevatedX scraper. Changed FCUK to ExploitedX network. Testing ElevatedX scraper with ExploitedX network. 2021-02-13 04:49:00 +01:00
DebaucheryLibrarian
a14227b588 Improved animated image handling. 2021-02-11 01:46:11 +01:00
DebaucheryLibrarian
42a2fd8800 Changed Dorcel trailer to teaser. 2021-02-10 23:49:37 +01:00
DebaucheryLibrarian
9aca5baa2b Added teaser support to Dorcel scraper. 2021-02-10 23:46:15 +01:00
DebaucheryLibrarian
7ff222ce25 Passing recursive parameters to all scraper methods. Using throttle parameters in MindGeek scraper, fixed missing slug breaking scene and actor URLs. 2021-02-10 03:23:48 +01:00
DebaucheryLibrarian
a04c7dda37 Added log to give insight about media insert failures. 2021-02-08 04:29:12 +01:00
DebaucheryLibrarian
dcad0cbe8f Fixed scenes without movies spawning empty movie objects and triggering 'missing entry ID' warning. 2021-02-08 03:35:24 +01:00
DebaucheryLibrarian
73b28866ac Scene tile uses first photo if poster is unavailable. 2021-02-08 02:10:56 +01:00
DebaucheryLibrarian
15cc970ee2 Added tags to release channels. Labeling DDF Network VR trailers as VR. 2021-02-05 01:54:06 +01:00
DebaucheryLibrarian
f217b161b4 Scraping scene photos from Bang API. 2021-02-04 22:55:19 +01:00
DebaucheryLibrarian
ff123b99b7 Added WankzVR update, scene and profile scraper. 2021-02-04 01:13:02 +01:00
DebaucheryLibrarian
0b99e72924 Added Kink VR to Kink network using BaDoink scraper. 2021-02-03 21:29:56 +01:00
DebaucheryLibrarian
79b51eca67 Added BaDoink profile scraper. Improved convert wrapper. 2021-02-03 21:03:35 +01:00
DebaucheryLibrarian
0c2e45141d Added BaDoink latest and scene scraper. 2021-02-03 02:59:39 +01:00
DebaucheryLibrarian
824fb9ef37 Changed profile network argument to context. 2021-02-03 00:50:00 +01:00
DebaucheryLibrarian
6d93083581 Removed superfluous MindGeek scrapers. 2021-02-03 00:46:59 +01:00