boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,853 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

758 Commits

Author SHA1 Message Date
DebaucheryLibrarian ff862dbff9 Removed debug log. 2021-02-23 04:10:03 +01:00
DebaucheryLibrarian fae288633c Catching actor association errors so it does not inhibit media association. 2021-02-23 04:09:33 +01:00
DebaucheryLibrarian f44cb8bf4c Merge branch 'master' into experimental 2021-02-23 03:36:56 +01:00
DebaucheryLibrarian c32a5d2930 Added debug log for actor entries. 2021-02-23 03:36:48 +01:00
DebaucheryLibrarian 2b5aac7633 Fixed S3 display support for movies. 2021-02-23 01:30:38 +01:00
DebaucheryLibrarian c1829c64c2 Not using media hash subdirs for S3 uploads. Updated video player for S3. 2021-02-23 00:54:19 +01:00
DebaucheryLibrarian 37e39dc1ec Added S3 support for media files. Fixed MindGeek scraper for new poster data structure. 2021-02-22 02:33:39 +01:00
DebaucheryLibrarian 9a65d8c0eb Merge branch 'master' into wasabi 2021-02-21 23:00:00 +01:00
DebaucheryLibrarian b79e75349c Fixed indentation. 2021-02-21 22:58:46 +01:00
DebaucheryLibrarian 8a739893ea Improved duplicate handling. Added tag photos. 2021-02-21 22:17:25 +01:00
DebaucheryLibrarian a39750d808 Fixed Nubiles deep scrape date. 2021-02-20 01:16:09 +01:00
DebaucheryLibrarian f310fec869 Added S3 experiment. 2021-02-20 01:12:44 +01:00
DebaucheryLibrarian 39e2abd80a Removed legacy comment. 2021-02-19 00:45:27 +01:00
DebaucheryLibrarian 5e2909c531 Fixed actor unique index preventing multiple actors with the same slug on a different network. Changed scene delete query to be able to handle more scene IDs. 2021-02-18 04:44:04 +01:00
DebaucheryLibrarian c51cd080fa Improved actor mapping in release associations. Storing alias ID in actor release association. 2021-02-17 00:40:20 +01:00
DebaucheryLibrarian 3469da674a Fixed PornCZ video query. 2021-02-16 19:53:32 +01:00
DebaucheryLibrarian 67055bf920 Improved actor entity and entry ID storage. 2021-02-16 03:37:52 +01:00
DebaucheryLibrarian 8a7baa02c1 Improved date query for upcoming scenes in ElevatedX scraper. 2021-02-14 14:31:33 +01:00
DebaucheryLibrarian 372b5da704 Moved all of ExploitedX and Nebraska Coeds into generic ElevatedX scraper. 2021-02-14 01:53:03 +01:00
DebaucheryLibrarian f79505f3f6 Added generic ElevatedX scraper. Changed FCUK to ExploitedX network. Testing ElevatedX scraper with ExploitedX network. 2021-02-13 04:49:00 +01:00
DebaucheryLibrarian a14227b588 Improved animated image handling. 2021-02-11 01:46:11 +01:00
DebaucheryLibrarian 42a2fd8800 Changed Dorcel trailer to teaser. 2021-02-10 23:49:37 +01:00
DebaucheryLibrarian 9aca5baa2b Added teaser support to Dorcel scraper. 2021-02-10 23:46:15 +01:00
DebaucheryLibrarian 7ff222ce25 Passing recursive parameters to all scraper methods. Using throttle parameters in MindGeek scraper, fixed missing slug breaking scene and actor URLs. 2021-02-10 03:23:48 +01:00
DebaucheryLibrarian a6c7c60290 Improved movie tile actor overflow. Improved deep movie error feedback. 2021-02-10 03:00:17 +01:00
DebaucheryLibrarian a04c7dda37 Added log to give insight about media insert failures. 2021-02-08 04:29:12 +01:00
DebaucheryLibrarian dcad0cbe8f Fixed scenes without movies spawning empty movie objects and triggering 'missing entry ID' warning. 2021-02-08 03:35:24 +01:00
DebaucheryLibrarian 73b28866ac Scene tile uses first photo if poster is unavailable. 2021-02-08 02:10:56 +01:00
DebaucheryLibrarian 80b8fe3654 Added actor flush, renamed inconsistent actor flush to actor delete. 2021-02-05 04:23:13 +01:00
DebaucheryLibrarian dbfbd1f04d Fixed movie flush confirmation. 2021-02-05 04:17:47 +01:00
DebaucheryLibrarian b94a3e05bc Added flush movie argument to index file. 2021-02-05 04:16:59 +01:00
DebaucheryLibrarian 80fa953f60 Added movie flush. 2021-02-05 04:14:13 +01:00
DebaucheryLibrarian a95a159978 Fixed movie scene column. 2021-02-05 04:06:53 +01:00
DebaucheryLibrarian 7b482e6903 Deleting movie scene associations before deleting scenes. 2021-02-05 04:05:44 +01:00
DebaucheryLibrarian 688863d16e Catching media storage errors to prevent full crash. 2021-02-05 03:37:04 +01:00
DebaucheryLibrarian 3e813ca251 Prevent writing to media hasher when hasher stream has closed. 2021-02-05 03:13:10 +01:00
DebaucheryLibrarian 15cc970ee2 Added tags to release channels. Labeling DDF Network VR trailers as VR. 2021-02-05 01:54:06 +01:00
DebaucheryLibrarian 5e12a1e1b1 Fixed not using media limit argument. 2021-02-04 22:58:18 +01:00
DebaucheryLibrarian f217b161b4 Scraping scene photos from Bang API. 2021-02-04 22:55:19 +01:00
DebaucheryLibrarian 98cae9270a Added video.js player with VR support for trailers. 2021-02-04 03:06:19 +01:00
DebaucheryLibrarian 315bf0fc89 Fixed missing date limit default argument. 2021-02-04 01:18:46 +01:00
DebaucheryLibrarian ff123b99b7 Added WankzVR update, scene and profile scraper. 2021-02-04 01:13:02 +01:00
DebaucheryLibrarian 0b99e72924 Added Kink VR to Kink network using BaDoink scraper. 2021-02-03 21:29:56 +01:00
DebaucheryLibrarian 79b51eca67 Added BaDoink profile scraper. Improved convert wrapper. 2021-02-03 21:03:35 +01:00
DebaucheryLibrarian 0c2e45141d Added BaDoink latest and scene scraper. 2021-02-03 02:59:39 +01:00
DebaucheryLibrarian 824fb9ef37 Changed profile network argument to context. 2021-02-03 00:50:00 +01:00
DebaucheryLibrarian 6d93083581 Removed superfluous MindGeek scrapers. 2021-02-03 00:46:59 +01:00
DebaucheryLibrarian 4db9a34a34 Removed superfluous Gamma scrapers. 2021-02-02 22:36:47 +01:00
DebaucheryLibrarian 3bbc5a5e87 Hard merging covers to preserve order. 2021-02-02 03:10:58 +01:00
DebaucheryLibrarian c1124abde0 Improved scene poster selector in Woodman scraper. 2021-02-02 02:18:12 +01:00
DebaucheryLibrarian 63f43013c3 Removed superfluous grandparent from entity query. 2021-02-02 02:11:16 +01:00
DebaucheryLibrarian e4e0eb23dd Removed unnecessary depth calculation from entity query. 2021-02-02 01:59:51 +01:00
DebaucheryLibrarian 6364912aa8 Added children to deep release entity, removed database dependency from Perfect Gonzo scraper. 2021-02-02 01:51:22 +01:00
DebaucheryLibrarian d5cdfb36a9 Selecting included networks with infinite parent depth to facilitate scraper resolve. 2021-02-02 01:31:12 +01:00
DebaucheryLibrarian 4b5cd50122 Fixed slug lookup in Perfect Gonzo scraper. 2021-02-01 20:49:08 +01:00
DebaucheryLibrarian aade7490f8 Querying infinite parent depth for deep release entities. 2021-02-01 01:45:30 +01:00
DebaucheryLibrarian 97c088cfb4 Added Anal Only and upcoming scraping to Mike Adriano. Fixed profile expand arrow color. 2021-01-30 17:43:33 +01:00
DebaucheryLibrarian bfb5006e95 Added actor scene URL parameter to Gamma scraper to phase out release URL function. 2021-01-30 01:12:42 +01:00
DebaucheryLibrarian 9535ab7953 Regarding layout parameters from parent and grandparent entities. Removed Evil Angel and Fantasy Massage scraper wrappers. 2021-01-30 00:01:40 +01:00
DebaucheryLibrarian 6938e88fbf Fixed some Gamma scene scrapers. 2021-01-29 04:26:45 +01:00
DebaucheryLibrarian b7aaeada45 Improved movie scraping. 2021-01-29 02:38:05 +01:00
DebaucheryLibrarian 4d89256a4c Merging improvements. 2021-01-27 00:21:58 +01:00
DebaucheryLibrarian b506a00e7d Improved scene merging. Improved Porn World/DDF scraper for poster and title redundancy. Fixed SFW poster showing in NSFW mode. 2021-01-25 23:53:56 +01:00
DebaucheryLibrarian b80eca35d8 Fixed Gamma scraper breaking when scene has no movie. Improved missing thumbnail presentation on mobile scene tiles. 2021-01-25 23:24:51 +01:00
DebaucheryLibrarian dc98fcad5a Scraping Gamma movies. Changed movie detail bar position, and scene detail bar mobile spacing. 2021-01-25 23:01:07 +01:00
DebaucheryLibrarian 486dbc5613 Improved Woodman scraper, added profiles. 2021-01-25 00:10:57 +01:00
DebaucheryLibrarian e9cbf5dab2 Added Woodman Casting X and WUNF update and scene scraper. Fixed actor and fallback media arrays being merged in deep scraper. 2021-01-24 23:31:28 +01:00
DebaucheryLibrarian 297f79f6e2 Restored sorting trailers by quality. 2021-01-24 20:43:34 +01:00
DebaucheryLibrarian 4151412156 Fixed Vixen trailer scraping. Using album instead of expand for actor photos. 2021-01-24 17:16:55 +01:00
DebaucheryLibrarian 985b523031 Scraping movie links from Gamma scene page. Removed expand option from scene page media to make way for album button. 2021-01-22 22:55:20 +01:00
DebaucheryLibrarian bd9b795516 Fixed avatars without entropy being discarded. 2021-01-22 15:40:49 +01:00
DebaucheryLibrarian b16cc26024 Added Zero Tolerance channel to network. 2021-01-22 00:26:06 +01:00
DebaucheryLibrarian a95e409366 Fixed Kink deep scrape photos. Fixed favicon ratio in compact scene tile. Hiding scroll buttons on small screens. 2021-01-19 15:58:27 +01:00
DebaucheryLibrarian 48f247a919 Changed scene media grid layout. 2021-01-17 02:07:02 +01:00
DebaucheryLibrarian 251bb9476d Added Gaywire, modified Bang Bros scraper to accomodate. 2021-01-17 01:43:55 +01:00
DebaucheryLibrarian e3ef0a0d69 Added Top Web Models profile scraper. 2021-01-16 04:10:43 +01:00
DebaucheryLibrarian b9e4764516 Fixed Pascals Sub Sluts interpreting metric as imperial height, filtering unlikely in interpolation. Splitting double actor entries in Top Web Models. 2021-01-15 16:14:48 +01:00
DebaucheryLibrarian b8df8e6507 Added Top Web Models update and scene scraper. 2021-01-15 04:04:32 +01:00
DebaucheryLibrarian 451ffdc48b Added Top Web Models directory. 2021-01-14 02:10:35 +01:00
DebaucheryLibrarian 12c3f0d7b9 Added Top Web Models framework. 2021-01-14 01:37:50 +01:00
DebaucheryLibrarian 39d149c728 Added default deep scrape fetch method. Added Karups scene and profile scraper. Added schoolgirl tag photo. 2021-01-13 21:29:05 +01:00
DebaucheryLibrarian 2bae6f693e Removed stray console log. 2021-01-13 16:08:53 +01:00
DebaucheryLibrarian ae0efccb04 Skipping Babel, updated node version. Improved deep scrape array merge. 2021-01-13 16:08:19 +01:00
DebaucheryLibrarian cdd4220cb9 Fixed release actor crash when no actors are present. 2021-01-13 15:43:53 +01:00
DebaucheryLibrarian f95c80f73f Using poster photo for Pinky XXX. 2021-01-13 01:10:29 +01:00
DebaucheryLibrarian 52356f0f31 Added Pinky XXX, changed ethnicity tags. 2021-01-13 00:52:05 +01:00
DebaucheryLibrarian ad73c11cb4 Using date and title for Nubiles entry ID. 2021-01-11 23:31:33 +01:00
DebaucheryLibrarian db4e74fb99 Fixed Nubiles base poster query, handling trailing commas in qu source set. Added profile scene scraper to Dogfart. Added tag photo. 2021-01-11 16:20:01 +01:00
DebaucheryLibrarian e38922f372 Removed redundant sitename from MindGeek session error. 2021-01-05 16:35:49 +01:00
DebaucheryLibrarian e1d6c9e489 Added site name to MindGeek session error. 2021-01-05 16:34:32 +01:00
DebaucheryLibrarian 9ca2ec6dd0 Fixed parent entity relations in seed file. Fixed MindGeek scraper session URL determination. 2021-01-05 16:27:20 +01:00
DebaucheryLibrarian 7bbb2f3557 Added tag filter dialog. 2021-01-03 22:53:51 +01:00
DebaucheryLibrarian 8739ec08cf Fixed qu init selector. Fixed Aziani scene page scope. 2021-01-02 03:20:39 +01:00
DebaucheryLibrarian 236d4fcde7 Fixed scene media lazy image. Fixed Perv City actor scope. 2021-01-01 04:28:10 +01:00
DebaucheryLibrarian ba3b87471e Added scene flush. Added temporary media insert log for integer out of range error. 2020-12-30 04:17:09 +01:00
DebaucheryLibrarian f0a90db912 Only allow actor flush by name for non-entity specific actors. 2020-12-30 03:39:40 +01:00
DebaucheryLibrarian 8aabcd6443 Added actor flush. 2020-12-30 03:19:09 +01:00
DebaucheryLibrarian af67d733ad Added profile flush. 2020-12-30 02:23:43 +01:00
DebaucheryLibrarian 770e5b75a5 Added profile interpolate command line argument. 2020-12-30 00:16:05 +01:00
DebaucheryLibrarian a8f68f4993 Fixed boolean handling in actor profile curation. 2020-12-29 23:44:38 +01:00
DebaucheryLibrarian 361e8f1bd3 Fixed Kelly Madison title regex. 2020-12-29 18:53:55 +01:00
DebaucheryLibrarian 0f052a0631 Fixed actor boolean logic, addressing missing boob, tattoo and piercing info. Removed scroll background, fixed actor photo load event and padding. 2020-12-29 04:20:33 +01:00
DebaucheryLibrarian bc944c2373 Restored Mike Adriano scraper. 2020-12-29 02:05:22 +01:00
DebaucheryLibrarian 8dd10f7e77 Adjusting tooltip arrow position, added open and close events. Fixed search tooltip layout. 2020-12-29 00:42:02 +01:00
DebaucheryLibrarian 6baa6b0802 Fixed fallback create dates in scene tiles. Fixed Mike Adriano entryIds and trailers for Nympho. 2020-12-28 01:29:34 +01:00
DebaucheryLibrarian 77ec2d3747 Using advanced merge library to maximize scraped data. 2020-12-27 23:43:08 +01:00
DebaucheryLibrarian 9c926a1d81 Fixed Mike Adriano dates for scene pages. 2020-12-27 23:36:15 +01:00
DebaucheryLibrarian ded414577f Fixed default actor avatar allocation. Fixed lazy loading in actor photos component. 2020-12-27 22:45:38 +01:00
DebaucheryLibrarian 27e5583849 Using generic session ID variable for to determine consent warning, rather than dedicated property. 2020-12-20 19:49:57 +01:00
DebaucheryLibrarian 5f4039c5d4 Added sharpness and re-added entropy to avatars, ignoring low-entropy photos as main avatar and in profile photo list. 2020-12-20 04:21:28 +01:00
DebaucheryLibrarian bd77d4347d Tied consent warning to session. 2020-12-19 23:01:17 +01:00
DebaucheryLibrarian be4d025505 Fixed trailer path regex for some Hush sites. 2020-12-19 02:35:41 +01:00
DebaucheryLibrarian 67ed249239 Added poster to API scene overview. 2020-12-19 01:03:15 +01:00
DebaucheryLibrarian 0d41fb48dc Removed outdated movie releases from API. Added API endpoint for scene posters. 2020-12-19 00:40:36 +01:00
DebaucheryLibrarian 4ccd8bf07a Fixed missing initial value in scraper reduce, breaking first network (21 Naturals). 2020-12-19 00:19:38 +01:00
DebaucheryLibrarian d67dca60fc Fixed birthdate interpolation and query. 2020-12-17 03:43:09 +01:00
DebaucheryLibrarian cd8e810c35 Fixed various Kelly Madison scraper issues. 2020-12-17 02:05:01 +01:00
DebaucheryLibrarian cc83b832f1 Attach base actor to scenes scraped from profile. 2020-12-16 00:50:58 +01:00
DebaucheryLibrarian 07f0249717 Upgraded pg, knex and postgraphile. Updated nvmrc version to latest node LTS. Fixed Gamma scraper not passing request headers to new http module. 2020-12-14 03:16:47 +01:00
DebaucheryLibrarian 71196688ae Removed console log. 2020-12-05 02:44:39 +01:00
DebaucheryLibrarian f0bec85ef8 Added Fuck'n'Drive and Jizz On Teens latest layouts to First Anal Quest scraper. 2020-12-05 02:24:31 +01:00
DebaucheryLibrarian 2e0fba3de9 Added First Anal Quest and Double View Casting profile scrapers. 2020-12-04 23:53:20 +01:00
DebaucheryLibrarian be1821b9eb Changed --inspect to --report to avoid conflict with Node's own debug tools. 2020-12-02 21:26:55 +01:00
DebaucheryLibrarian 2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian 6f19a8a642 Using ageFromBirth for age on scene date. Using infinite depth and colors for inspect. 2020-12-01 23:00:14 +01:00
DebaucheryLibrarian 9a61d2305c Added fixed actor age. Added male profiles to Littlr Caprice Dreams scraper. Added various tag photos. 2020-11-29 03:59:47 +01:00
DebaucheryLibrarian 71c884fe48 Improved Little Caprice Dreams scraper. 2020-11-28 00:46:30 +01:00
DebaucheryLibrarian 9a183c7ffb Added channel matching to Little Caprice Dreams. 2020-11-27 03:23:12 +01:00
DebaucheryLibrarian 60485751e2 Added support for upcoming scenes to Bang scraper. 2020-11-26 22:07:54 +01:00
DebaucheryLibrarian 4e559f63e3 Including all children of included networks, separated included children into dedicated property. 2020-11-26 04:26:52 +01:00
DebaucheryLibrarian 1b407254a7 Increased hard limit in release API. 2020-11-26 04:01:01 +01:00
DebaucheryLibrarian 0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian 6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian 54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian 980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian 711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian 8688c28d0f Replaced queueMethod in media sources for new interval/concurrency options. 2020-11-23 00:38:22 +01:00
DebaucheryLibrarian ca7c8f0afd Updated place resolve module to use new HTTP module. 2020-11-23 00:30:20 +01:00
DebaucheryLibrarian 0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian 3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian 6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian 081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian 77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian 8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian 2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00