boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,056 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

834 Commits

Author SHA1 Message Date
DebaucheryLibrarian 251bb9476d Added Gaywire, modified Bang Bros scraper to accomodate. 2021-01-17 01:43:55 +01:00
DebaucheryLibrarian e3ef0a0d69 Added Top Web Models profile scraper. 2021-01-16 04:10:43 +01:00
DebaucheryLibrarian b9e4764516 Fixed Pascals Sub Sluts interpreting metric as imperial height, filtering unlikely in interpolation. Splitting double actor entries in Top Web Models. 2021-01-15 16:14:48 +01:00
DebaucheryLibrarian b8df8e6507 Added Top Web Models update and scene scraper. 2021-01-15 04:04:32 +01:00
DebaucheryLibrarian 451ffdc48b Added Top Web Models directory. 2021-01-14 02:10:35 +01:00
DebaucheryLibrarian 12c3f0d7b9 Added Top Web Models framework. 2021-01-14 01:37:50 +01:00
DebaucheryLibrarian 39d149c728 Added default deep scrape fetch method. Added Karups scene and profile scraper. Added schoolgirl tag photo. 2021-01-13 21:29:05 +01:00
DebaucheryLibrarian 2bae6f693e Removed stray console log. 2021-01-13 16:08:53 +01:00
DebaucheryLibrarian ae0efccb04 Skipping Babel, updated node version. Improved deep scrape array merge. 2021-01-13 16:08:19 +01:00
DebaucheryLibrarian cdd4220cb9 Fixed release actor crash when no actors are present. 2021-01-13 15:43:53 +01:00
DebaucheryLibrarian f95c80f73f Using poster photo for Pinky XXX. 2021-01-13 01:10:29 +01:00
DebaucheryLibrarian 52356f0f31 Added Pinky XXX, changed ethnicity tags. 2021-01-13 00:52:05 +01:00
DebaucheryLibrarian ad73c11cb4 Using date and title for Nubiles entry ID. 2021-01-11 23:31:33 +01:00
DebaucheryLibrarian db4e74fb99 Fixed Nubiles base poster query, handling trailing commas in qu source set. Added profile scene scraper to Dogfart. Added tag photo. 2021-01-11 16:20:01 +01:00
DebaucheryLibrarian e38922f372 Removed redundant sitename from MindGeek session error. 2021-01-05 16:35:49 +01:00
DebaucheryLibrarian e1d6c9e489 Added site name to MindGeek session error. 2021-01-05 16:34:32 +01:00
DebaucheryLibrarian 9ca2ec6dd0 Fixed parent entity relations in seed file. Fixed MindGeek scraper session URL determination. 2021-01-05 16:27:20 +01:00
DebaucheryLibrarian 7bbb2f3557 Added tag filter dialog. 2021-01-03 22:53:51 +01:00
DebaucheryLibrarian 8739ec08cf Fixed qu init selector. Fixed Aziani scene page scope. 2021-01-02 03:20:39 +01:00
DebaucheryLibrarian 236d4fcde7 Fixed scene media lazy image. Fixed Perv City actor scope. 2021-01-01 04:28:10 +01:00
DebaucheryLibrarian ba3b87471e Added scene flush. Added temporary media insert log for integer out of range error. 2020-12-30 04:17:09 +01:00
DebaucheryLibrarian f0a90db912 Only allow actor flush by name for non-entity specific actors. 2020-12-30 03:39:40 +01:00
DebaucheryLibrarian 8aabcd6443 Added actor flush. 2020-12-30 03:19:09 +01:00
DebaucheryLibrarian af67d733ad Added profile flush. 2020-12-30 02:23:43 +01:00
DebaucheryLibrarian 770e5b75a5 Added profile interpolate command line argument. 2020-12-30 00:16:05 +01:00
DebaucheryLibrarian a8f68f4993 Fixed boolean handling in actor profile curation. 2020-12-29 23:44:38 +01:00
DebaucheryLibrarian 361e8f1bd3 Fixed Kelly Madison title regex. 2020-12-29 18:53:55 +01:00
DebaucheryLibrarian 0f052a0631 Fixed actor boolean logic, addressing missing boob, tattoo and piercing info. Removed scroll background, fixed actor photo load event and padding. 2020-12-29 04:20:33 +01:00
DebaucheryLibrarian bc944c2373 Restored Mike Adriano scraper. 2020-12-29 02:05:22 +01:00
DebaucheryLibrarian 8dd10f7e77 Adjusting tooltip arrow position, added open and close events. Fixed search tooltip layout. 2020-12-29 00:42:02 +01:00
DebaucheryLibrarian 6baa6b0802 Fixed fallback create dates in scene tiles. Fixed Mike Adriano entryIds and trailers for Nympho. 2020-12-28 01:29:34 +01:00
DebaucheryLibrarian 77ec2d3747 Using advanced merge library to maximize scraped data. 2020-12-27 23:43:08 +01:00
DebaucheryLibrarian 9c926a1d81 Fixed Mike Adriano dates for scene pages. 2020-12-27 23:36:15 +01:00
DebaucheryLibrarian ded414577f Fixed default actor avatar allocation. Fixed lazy loading in actor photos component. 2020-12-27 22:45:38 +01:00
DebaucheryLibrarian 27e5583849 Using generic session ID variable for to determine consent warning, rather than dedicated property. 2020-12-20 19:49:57 +01:00
DebaucheryLibrarian 5f4039c5d4 Added sharpness and re-added entropy to avatars, ignoring low-entropy photos as main avatar and in profile photo list. 2020-12-20 04:21:28 +01:00
DebaucheryLibrarian bd77d4347d Tied consent warning to session. 2020-12-19 23:01:17 +01:00
DebaucheryLibrarian be4d025505 Fixed trailer path regex for some Hush sites. 2020-12-19 02:35:41 +01:00
DebaucheryLibrarian 67ed249239 Added poster to API scene overview. 2020-12-19 01:03:15 +01:00
DebaucheryLibrarian 0d41fb48dc Removed outdated movie releases from API. Added API endpoint for scene posters. 2020-12-19 00:40:36 +01:00
DebaucheryLibrarian 4ccd8bf07a Fixed missing initial value in scraper reduce, breaking first network (21 Naturals). 2020-12-19 00:19:38 +01:00
DebaucheryLibrarian d67dca60fc Fixed birthdate interpolation and query. 2020-12-17 03:43:09 +01:00
DebaucheryLibrarian cd8e810c35 Fixed various Kelly Madison scraper issues. 2020-12-17 02:05:01 +01:00
DebaucheryLibrarian cc83b832f1 Attach base actor to scenes scraped from profile. 2020-12-16 00:50:58 +01:00
DebaucheryLibrarian 07f0249717 Upgraded pg, knex and postgraphile. Updated nvmrc version to latest node LTS. Fixed Gamma scraper not passing request headers to new http module. 2020-12-14 03:16:47 +01:00
DebaucheryLibrarian 71196688ae Removed console log. 2020-12-05 02:44:39 +01:00
DebaucheryLibrarian f0bec85ef8 Added Fuck'n'Drive and Jizz On Teens latest layouts to First Anal Quest scraper. 2020-12-05 02:24:31 +01:00
DebaucheryLibrarian 2e0fba3de9 Added First Anal Quest and Double View Casting profile scrapers. 2020-12-04 23:53:20 +01:00
DebaucheryLibrarian be1821b9eb Changed --inspect to --report to avoid conflict with Node's own debug tools. 2020-12-02 21:26:55 +01:00
DebaucheryLibrarian 2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian 6f19a8a642 Using ageFromBirth for age on scene date. Using infinite depth and colors for inspect. 2020-12-01 23:00:14 +01:00
DebaucheryLibrarian 9a61d2305c Added fixed actor age. Added male profiles to Littlr Caprice Dreams scraper. Added various tag photos. 2020-11-29 03:59:47 +01:00
DebaucheryLibrarian 71c884fe48 Improved Little Caprice Dreams scraper. 2020-11-28 00:46:30 +01:00
DebaucheryLibrarian 9a183c7ffb Added channel matching to Little Caprice Dreams. 2020-11-27 03:23:12 +01:00
DebaucheryLibrarian 60485751e2 Added support for upcoming scenes to Bang scraper. 2020-11-26 22:07:54 +01:00
DebaucheryLibrarian 4e559f63e3 Including all children of included networks, separated included children into dedicated property. 2020-11-26 04:26:52 +01:00
DebaucheryLibrarian 1b407254a7 Increased hard limit in release API. 2020-11-26 04:01:01 +01:00
DebaucheryLibrarian 0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian 6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian 54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian 980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian 711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian 8688c28d0f Replaced queueMethod in media sources for new interval/concurrency options. 2020-11-23 00:38:22 +01:00
DebaucheryLibrarian ca7c8f0afd Updated place resolve module to use new HTTP module. 2020-11-23 00:30:20 +01:00
DebaucheryLibrarian 0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian 3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian 6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian 081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian 77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian 8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian 2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00
DebaucheryLibrarian 8586817963 Fixed Insex html table selector. 2020-11-01 03:25:30 +01:00
DebaucheryLibrarian c1fffe5cdb Added alternative layout to Insex for updated Topgrl and Sexually Broken sites. 2020-11-01 02:58:21 +01:00
DebaucheryLibrarian 3855d96135 Removed 'null' from scene tile URL if no slug is available. 2020-10-30 17:51:47 +01:00
DebaucheryLibrarian 39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian c37d4ad01f Filtering invalid actors from releases before storing. 2020-10-29 16:06:20 +01:00
DebaucheryLibrarian b188bc5744 Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability. 2020-10-29 15:20:59 +01:00
DebaucheryLibrarian 4469376dd2 Using temporary table instead of WHERE IN to stack depth error when finding duplicate actors. 2020-10-28 03:50:52 +01:00
DebaucheryLibrarian bf9b334b73 Adding scraper config by scraper slug to current 'includes' parameter. 2020-10-28 01:36:13 +01:00
DebaucheryLibrarian 1869877178 Added release covers table to flush routine. 2020-10-27 02:40:30 +01:00
DebaucheryLibrarian 7257776ba8 Mapping avatar to media ID in orphan delete. 2020-10-27 02:34:06 +01:00
DebaucheryLibrarian 1d3ec96e8d Detecting profile avatars in orphaned media flush. 2020-10-27 02:09:52 +01:00
DebaucheryLibrarian 99a7bfeb4c Added traxxx dummy network. 2020-10-26 01:42:38 +01:00
DebaucheryLibrarian 0027be65eb Fixed CzechAV to accomodate teasers instead of photos. Fixed error trying to flush non-existent media file. 2020-10-25 21:43:36 +01:00
DebaucheryLibrarian 57a44d6643 Expanded README. 2020-10-25 01:29:25 +02:00
DebaucheryLibrarian 0bd7fca876 Added orphaned media flush and batch release flush. 2020-10-25 00:52:40 +02:00
DebaucheryLibrarian 6791053c83 Fixed entity alias available through wrong type endpoint. 2020-10-20 21:04:29 +02:00
DebaucheryLibrarian 9f3c686913 Added thumbnail and favicon to entity REST API. 2020-10-20 15:37:42 +02:00
DebaucheryLibrarian ecdd6d8fb0 Added logo path to entity API. 2020-10-20 15:28:58 +02:00
DebaucheryLibrarian 60eb599416 Added alias to entity search query. 2020-10-20 00:25:32 +02:00
DebaucheryLibrarian 6b17f9d1f2 Allowing entity to be fetched by alias. 2020-10-20 00:21:15 +02:00
DebaucheryLibrarian d795266114 Removed no-date-limit as argument due yargs conflict. 2020-10-20 00:08:14 +02:00
DebaucheryLibrarian 2f8fca0327 Added missing-date as config and argument alias. 2020-10-20 00:05:23 +02:00
DebaucheryLibrarian 4a900cbbeb Renamed nullDateLimit to noDateLimit in config. 2020-10-20 00:03:22 +02:00
DebaucheryLibrarian 5bdbb5ec62 Renamed null-date-limit to no-date-limit, added old as alias. 2020-10-20 00:01:29 +02:00
DebaucheryLibrarian 593ce27312 Added rudimentary scene and entity scene remove. 2020-10-19 02:02:21 +02:00
DebaucheryLibrarian 2536405dba Fixed entity API database query. 2020-10-18 00:01:34 +02:00
DebaucheryLibrarian ca22aedaaa Added rudimentary API documentation to README. 2020-10-17 22:54:00 +02:00
DebaucheryLibrarian e6c52002f0 Added tags and entities to REST API.. 2020-10-16 23:00:03 +02:00
DebaucheryLibrarian f38233053e Removed debug log. 2020-10-15 01:45:13 +02:00
DebaucheryLibrarian 99a4751c20 Returning results from new pagination. 2020-10-14 03:17:03 +02:00
DebaucheryLibrarian 7c856c267d Added revised next page determination. 2020-10-12 04:08:22 +02:00
DebaucheryLibrarian 8aefb8eddb Added and updated tag photos. 2020-10-07 03:40:19 +02:00
DebaucheryLibrarian 3f843cc0fc Added version to stats page. 2020-09-25 21:21:26 +02:00
DebaucheryLibrarian 566c20ea7e Added various tag photos. Renamed some toy tags. 2020-09-21 05:11:24 +02:00
DebaucheryLibrarian e78bfe4c22 Upgraded knex and pg versions. 2020-09-19 00:12:15 +02:00
DebaucheryLibrarian c94e1aaea9 Handling missing trailers in Kink scraper. 2020-09-18 23:25:15 +02:00
DebaucheryLibrarian 3789ef51f2 (Temporarily) removed studio filter from entity query for performance reasons. 2020-09-18 22:43:45 +02:00
DebaucheryLibrarian 3c9468b0f1 Fixed wrong MindGeek session acquire URL. 2020-09-18 03:27:00 +02:00
DebaucheryLibrarian a4929819df Using channel URL instead of composed URL for session retrieval, should fix Brazzers. 2020-09-18 02:54:05 +02:00
DebaucheryLibrarian a9fa71e455 Fixed predata parameter in Assylum scraper. 2020-09-17 14:49:45 +02:00
DebaucheryLibrarian 1a8de4fcf6 Added mimetype verification option to media source to ensure server returned a plausible file. Added additional fallbacks to Jules Jordan poster scraper for Amateur Allure. 2020-09-17 02:30:58 +02:00
DebaucheryLibrarian 0190ee9531 Fixed Porn Doe's poster query. Checking style attribute existence in qu before attempting to use it. 2020-09-17 00:35:41 +02:00
DebaucheryLibrarian 6fb15fb591 Added Dorcel Club with scene and actor scraping. Added count method to qu. 2020-09-16 04:55:30 +02:00
DebaucheryLibrarian a8c525f4fc Hard-coded Pascal White as the male actor for Pascal's Sub Sluts. Added female gender to all Sub Sluts. 2020-09-16 02:38:59 +02:00
DebaucheryLibrarian 7c4dd03a8c Ignoring 'Lockdown Submissions' as actor for Pascals Subsluts. 2020-09-16 01:45:58 +02:00
DebaucheryLibrarian 7fd7005776 Added profile scraper to Pascals Subsluts. 2020-09-16 01:42:15 +02:00
DebaucheryLibrarian bc34c6edb4 Added Bang Bros Vault channel. 2020-09-14 16:01:27 +02:00
DebaucheryLibrarian ba7419d3b0 Added basic Pascals Subsluts scraper. 2020-09-14 02:40:27 +02:00
DebaucheryLibrarian beeaebbfb7 Added CzechAV. 2020-09-14 00:53:41 +02:00
DebaucheryLibrarian b57b0a38f5 Fixed infinite pagination when scraping upcoming scenes. 2020-09-13 01:41:52 +02:00
DebaucheryLibrarian eb6337f6fb Fixed upcoming Jules Jordan scene without teaser breaking scraper. 2020-09-11 02:36:36 +02:00
DebaucheryLibrarian 0e8024adf1 Changed next page determination to ensure --after is followed even if there are no unique releases. 2020-09-10 23:49:24 +02:00
DebaucheryLibrarian bec097f14d Returning status codes from Gamma fetch. 2020-09-10 17:54:23 +02:00
DebaucheryLibrarian 7f8704ee2c Using http instead of bhttp for Gamma. 2020-09-10 17:41:29 +02:00
DebaucheryLibrarian 5ee5b270ef Added avatars to Gamma latest API. 2020-09-10 03:56:09 +02:00
DebaucheryLibrarian 88eeab410b Added filter parameter to Gamma scraper to distinguish Evil Angel exclusives from channel scenes. 2020-09-10 03:43:16 +02:00
DebaucheryLibrarian 0d4893b13c Added Evil Angel channels with logos. 2020-09-10 03:17:19 +02:00
DebaucheryLibrarian ac66606135 Including networks in filter count. Preserving query between date ranges. Allowing --latest to be used without --last. 2020-09-09 03:45:00 +02:00
DebaucheryLibrarian 8aaa88770f Accounting for Windows paths in logger. 2020-09-08 15:44:55 +02:00
DebaucheryLibrarian 03ba35d65a Fixed include object. Fixed qu's undefined URL handling. 2020-09-08 03:26:34 +02:00
DebaucheryLibrarian e90bb63a8f Added American Pornstar. Improved Jules Jordan scraper to accomodate for American Pornstar. Changed entity logo mogrify settings to ensure both minimum height and width. 2020-09-05 01:56:54 +02:00
DebaucheryLibrarian 3c84a814a8 Added Zero Tolerance with Addicted 2 Girls and GenderX. 2020-09-04 03:07:28 +02:00
DebaucheryLibrarian c187a27123 Added Hookup Hotshot. 2020-09-03 22:22:12 +02:00
DebaucheryLibrarian f6353ca14c Storing actor profile URL when provided from scene page. 2020-08-31 02:43:41 +02:00
DebaucheryLibrarian 1bfdf4b232 Storing actor profiles from scene pages. 2020-08-30 04:18:47 +02:00
DebaucheryLibrarian 53b0101a12 Added new Naughty America sites. 2020-08-26 23:59:29 +02:00
DebaucheryLibrarian 8611d738b0 Using UTC to query date ranges. Removed stray console log from MindGeek scraper. 2020-08-26 02:01:38 +02:00
DebaucheryLibrarian 52f66e7982 Fixed undefined location in FreeOnes scraper. 2020-08-24 18:24:07 +02:00
DebaucheryLibrarian 7fed5b7138 Moved Brazzers to MindGeek scraper to support new site. 2020-08-24 05:13:34 +02:00
DebaucheryLibrarian d8b8dfa299 Not creating batch ID when no movies are to be stored. 2020-08-23 03:41:29 +02:00
DebaucheryLibrarian c3d771c8fc Hush scraper uses children from entity argument for filter regexp, instead of making its own database request. 2020-08-23 03:31:37 +02:00
DebaucheryLibrarian 278b74e78c Providing duplicate releases in predata. Using duplicates for filtering scenes without channel in Hush scraper. 2020-08-23 02:43:10 +02:00
DebaucheryLibrarian 3a5ea3dd9a Returning duplicate releases from pagination. 2020-08-22 04:22:56 +02:00
DebaucheryLibrarian 5f5c48ea05 Fixed pagination behavior for upcoming scenes. 2020-08-22 01:57:23 +02:00
DebaucheryLibrarian 7c052fedfc Fixed map error. 2020-08-22 00:28:22 +02:00
DebaucheryLibrarian 1c17cd1be5 Improved update scrape pagination and limits. 2020-08-21 03:55:51 +02:00
DebaucheryLibrarian 4ec89e2cc8 Added upcoming, profile and detailed scene actor scraping to InTheCrack. Fixed clip upsert. 2020-08-20 23:35:18 +02:00
DebaucheryLibrarian 501e764c21 Renamed chapters to clips. Fixed Vixen trailers. 2020-08-20 19:52:02 +02:00
DebaucheryLibrarian 2835c66694 Added chapters and shoot location. Added In The Crack. 2020-08-20 04:57:38 +02:00
DebaucheryLibrarian fd4477bc50 Improved 'new' sorting. 2020-08-19 21:48:55 +02:00
DebaucheryLibrarian e896d52968 Added separate task queue for video streams to prevent ffmpeg overstressing the CPU. Fixed entity parent in scene REST API. 2020-08-17 15:53:20 +02:00
DebaucheryLibrarian b3435c97c3 Added footer and basic stats page. 2020-08-15 19:04:33 +02:00
DebaucheryLibrarian d7974f057f Fixed scene tile detail word wrap. Fixed Score posters. 2020-08-15 02:05:35 +02:00
DebaucheryLibrarian 50c5f921f5 Using new bulk insert utility for releases, media and actors. 2020-08-14 23:21:53 +02:00
DebaucheryLibrarian e996a45bf5 Added new bulk upsert utility. 2020-08-14 23:05:25 +02:00
DebaucheryLibrarian b3f784686f Improved entity provision behavior. 2020-08-14 00:32:59 +02:00
DebaucheryLibrarian 77566eae0d Fixed and documented entity configuration and query. 2020-08-13 23:59:54 +02:00
DebaucheryLibrarian 59e2124407 Removed type property from scenes API. 2020-08-13 16:10:58 +02:00
DebaucheryLibrarian f8c9b69f4b Allowing --actors-update to be used without --actors. 2020-08-12 21:04:38 +02:00
DebaucheryLibrarian d14034d38b Defauling --actors-update to 1900-01-01 2020-08-12 21:00:50 +02:00
DebaucheryLibrarian 7413d7db25 Improved and documented actor profile scraping. 2020-08-12 20:51:08 +02:00
DebaucheryLibrarian 5cabeed19d Modularized release component between movie and scene. Added Kink Classics channel. 2020-08-12 03:30:20 +02:00
DebaucheryLibrarian 40aed1086f Showing actors, tags and date on movie tiles. 2020-08-10 21:39:55 +02:00
DebaucheryLibrarian a7d5bef93f Filtering undefined scenes property from movies. Added movie page scraper to Elegant Angel. 2020-08-08 18:10:59 +02:00
DebaucheryLibrarian b4f0501765 Scraping from Cherry Pimps when available. Showing cover in movie tile. 2020-08-02 03:44:14 +02:00
DebaucheryLibrarian 767437d9aa Added movie tile. Fixed actor header. Larger breakpoint for nav menu. 2020-08-01 15:11:07 +02:00
DebaucheryLibrarian 6c5a62353c Moved movies to separate table. 2020-07-25 03:44:19 +02:00
DebaucheryLibrarian bfd54e94e7 Improved Team Skeet profile scraper. 2020-07-23 21:00:16 +02:00
DebaucheryLibrarian 747c2e1637 Checking nationality against alpha2 and alpha2. Improved Team Skeet profile scraper. 2020-07-23 04:39:12 +02:00
DebaucheryLibrarian 23e4f87af0 Added profile scraping and Hoby Buchanon to Team Skeet. 2020-07-23 04:29:46 +02:00
DebaucheryLibrarian 9ef5ea8fb6 Added the awkward pagination to PornCZ scraper. 2020-07-23 00:55:55 +02:00
DebaucheryLibrarian 46c6c4dd21 Added PornCZ. 2020-07-22 04:12:20 +02:00
DebaucheryLibrarian 9d89a38490 Added actor page scene thumbnails to Hitzefrei scraper. 2020-07-21 04:10:16 +02:00
DebaucheryLibrarian d56da74168 Added Hitzefrei. Fixed date averaging. 2020-07-21 04:04:07 +02:00
DebaucheryLibrarian dff4d15872 Updated profile scrapers to use base actor instead of actor name. Fixes for Reality Kings and Cherry Pimps scrapers. 2020-07-21 01:44:51 +02:00
DebaucheryLibrarian 939eba8e61 Changed qu's HTML element detection. Passing base actor instead of actorName to profile scrapers. 2020-07-21 01:16:26 +02:00
DebaucheryLibrarian 5291a87587 Added 'clear all' button to tag filter. Tag name never removes tag. Added actor scene scraping to LegalPorno. 2020-07-19 19:40:21 +02:00
DebaucheryLibrarian 689c701f34 Various tag photos. 2020-07-18 05:12:32 +02:00
DebaucheryLibrarian a3d281192d Reordered scraper arguments. Fixed Jules Jordan scraper for Amateur Allure. 2020-07-17 23:27:59 +02:00
DebaucheryLibrarian f59e809713 Added experimental movie page scraping with Elegant Angel. 2020-07-17 04:33:05 +02:00
DebaucheryLibrarian a88c2f0760 Added m3u8 stream support to media module. Added Elegant Angel. Added regex parameter to qu's number method. Various tags. 2020-07-17 03:39:13 +02:00
DebaucheryLibrarian 66d6322c1d Updated GraphQL queries to Datetime. Updated template to latest guideline. 2020-07-16 15:55:03 +02:00
DebaucheryLibrarian 6adfded074 Refactored Mike Adriano scraper. Changed logo and favicon. Added style methods to qu. 2020-07-16 03:47:07 +02:00
DebaucheryLibrarian 5b886b3917 Improved actor extraction for fcuk scraper. Changed 'copyright' to 'credit'. Redused entity page favicon size. 2020-07-15 05:12:29 +02:00
DebaucheryLibrarian c62df2228b Added scraper for FCUK's coed sites. 2020-07-15 04:51:39 +02:00
DebaucheryLibrarian 17b3ba1272 Added partial 'fcuk' (Exploited College Girls) scraper. Added file parameter for actor names and scene URLs. 2020-07-15 03:24:47 +02:00
DebaucheryLibrarian 90fc7a0d9d Fixed Kink profile scraper returning partial matches. Removed parent from BAM Visions and Vogov. 2020-07-14 21:21:27 +02:00
DebaucheryLibrarian 74b15aa8e9 Added 5K Porn and 5K Teens. 2020-07-14 04:36:14 +02:00
DebaucheryLibrarian b7be97fcf9 Added Amateur Euro, For Bondage, Mamacitaz, TransBella and VIP Sex Vault. Refactored Kelly Madison scraper using qu, fixed trailers and improved reliability. 2020-07-14 03:46:31 +02:00
ThePendulum 1eab3be7f6 Added Lets Doe It scraper. Added timestamp matching to qu's duration method. 2020-07-13 03:51:17 +02:00
ThePendulum 6fd2bc2687 Added Kink profile scraper. Fixed --force causing media collisions. 2020-07-13 00:12:01 +02:00
ThePendulum a7707b7b28 Merged DDF Network with Porn World. Fixed and updated DDF/PW scraper. 2020-07-12 22:36:53 +02:00
ThePendulum 70c60e93ac Scraping Perv City updates to network entity. 2020-07-12 05:25:27 +02:00
ThePendulum 9c8cfe3bdb Re-wrote broken Perv City scraper, added profile scraping. 2020-07-12 05:10:23 +02:00
ThePendulum e8c55512e2 Added image processing options to media module. Cropping Killergram avatars. Overwriting images when --force is used. 2020-07-10 03:42:08 +02:00
ThePendulum 4c551cc15f Fixed pagination for Killergram, added pagination to actor profiles, added Killergram Platinum. Added experimental m3u8 stream support. 2020-07-10 02:01:23 +02:00
ThePendulum a013d73400 Added Killergram. 2020-07-09 04:31:27 +02:00
ThePendulum 44a8ced30c Separated actor expand buttons. Refactored Brazzers scraper. Fixed actor releases not included in shallow scrape. Added number query and data-src default to qu img. Updated README. Removed post-install migrate and seed. 2020-07-09 02:00:54 +02:00
ThePendulum 17d46e804e Updated tag photos. 2020-07-08 04:58:12 +02:00
ThePendulum 9e5d3aa139 Added 'visible' property to entities. Added various Teen Core Club logos. 2020-07-08 02:53:46 +02:00
ThePendulum 0fd1edd8fd Added Teen Core Club profile scraper. Added all TCC channels. Added various TCC logos. 2020-07-07 04:37:12 +02:00
ThePendulum abe7de5701 Added scene scraping to Teen Core Club. 2020-07-06 04:13:48 +02:00
ThePendulum f4ef2d4cc2 Auto-shrinking pagination. Various Teen Core Club curations. 2020-07-05 04:10:35 +02:00
ThePendulum c47fae3a1b Setting composed title as page title. Using Teen Core Club title if it isn't shoot ID. 2020-07-03 04:42:20 +02:00
ThePendulum 48a127409e Added date precision. Added Teen Core Club update scraper. 2020-07-03 04:12:56 +02:00
ThePendulum 945c2c45ce Updated tag page layout. Added shoot date property. Showing parent favicon on compact entity page. Re-added 'new' indicator on tile. Added Family Sinner to Mile High Media. Various fixes and improvements. 2020-07-03 01:28:22 +02:00
ThePendulum 749864e922 Added 'independent' property for network-like channels. Changed release tile design. Adding Brazzers timeline events as tags. Added Property Sex to MindGeek. Changed DP, DAP and DVP tag slugs. Changed Porn Pros logo. Added better BAM Visions and Mug Fucked logos. 2020-07-02 04:04:28 +02:00
ThePendulum 1f444e58ce Allowing image sources to specify queue method. Using 5s queue for Whale Member to avoid CDN time-outs. 2020-07-01 04:47:05 +02:00
ThePendulum 53870fda89 Improved release detail bar behavior. 2020-07-01 00:25:27 +02:00
ThePendulum ff384fb734 Fixed search documents to coalesce empty parent entities. 2020-06-30 01:52:17 +02:00
ThePendulum 98c19b560f Updated mindgeek scraper for entities. Various fixes. 2020-06-28 22:29:18 +02:00
ThePendulum 3462d7af2a Entity refactor. Facilitating channels without parent. 2020-06-28 00:15:13 +02:00
ThePendulum af56378ee2 Refactored various modules for entities. Updated and refactored Kink scraper. 2020-06-27 02:57:30 +02:00
ThePendulum 4959dfd14f Refactored deep and store modules to use entities. 2020-06-25 02:26:25 +02:00
ThePendulum f0a89df6ab Refactoring to use entities over sites and networks. 2020-06-17 04:07:24 +02:00
ThePendulum 1907ce1e54 Changed sites from argument query to group by network. 2020-06-15 03:58:35 +02:00
ThePendulum 79465d9634 Added Teen Core Club. Changed network to entity in GraphQL query. 2020-06-08 03:41:12 +02:00
ThePendulum 09d849eb9d Adding networks and sites as entities, 2020-06-04 01:03:02 +02:00
ThePendulum 8abcc7194a Updated Full Porn Network scraper. 2020-05-29 22:43:03 +02:00
ThePendulum 88a88227c4 Applying tag filters to URL. 2020-05-27 01:40:10 +02:00
ThePendulum 86377fec5f Enabled pagination on network page. 2020-05-26 04:11:29 +02:00
ThePendulum 9843023c1f Added Filthy Family and 'legacy' scraper to Bang Bros. Added trans generic avatar. Added pagination support to site actions. 2020-05-24 03:54:29 +02:00
ThePendulum 75d49517b7 Added pagination to actor overview. Lazy loading actor avatars. Reduced hash digest length. 2020-05-23 04:32:50 +02:00
ThePendulum 2fcd426b49 Added basic pagination to homepage. 2020-05-22 04:32:16 +02:00
ThePendulum f38be7a706 Added components. 2020-05-22 00:55:11 +02:00
ThePendulum 532a4b679b Using batch insert for profiles to prevent errors on large inserts. 2020-05-21 03:44:44 +02:00
ThePendulum 703b77897d Added actor scenes to deep fetch sources. 2020-05-20 03:03:42 +02:00
ThePendulum 7275fb10e3 Improved Gamma scraper for (XEmpire) actors. 2020-05-20 03:00:46 +02:00
ThePendulum b1b7cd6d50 Fixed Whale Member posters and photos. 2020-05-20 02:23:45 +02:00
ThePendulum b6691e1991 Added release type distinction to REST API. 2020-05-20 01:38:58 +02:00
ThePendulum 057362d011 Added basic release and actor API. 2020-05-20 01:11:32 +02:00
ThePendulum 6973d39cbd Replacing one or multiple space-like characters with single space in actor description. Using logo thumbnails in descriptions. 2020-05-19 05:06:01 +02:00
ThePendulum c0898b84d6 Showing all unique descriptions on profile with network logo. Fixed Fame Digital scraper. 2020-05-19 04:46:49 +02:00
ThePendulum 9883c3d9c2 Not scraping existing actor profiles unless --force is used. 2020-05-19 02:02:48 +02:00
ThePendulum 0c4628677f Fixed profile location interpolation. Generalizing ethnicity, hair color and eye color. 2020-05-19 01:10:32 +02:00
ThePendulum 4826ae8571 Added oil tag. Removed Fame Digital from profile config. 2020-05-18 04:28:38 +02:00
ThePendulum 003e07491d Added network to profile context when site is available. 2020-05-18 03:34:41 +02:00
ThePendulum 885aa4f627 Passing context object with site or network instead of scraper slug and 'site or network' to all profile scrapers. 2020-05-18 03:22:03 +02:00
ThePendulum 8733fdc657 Improved actor scraping and display. 2020-05-18 01:22:56 +02:00
ThePendulum af5543190a Showing avatar on release actor tile. Fixed social URL validation. 2020-05-17 05:24:46 +02:00
ThePendulum 61965d480c Added birthday to Bang actor scraper. 2020-05-17 05:08:41 +02:00
ThePendulum f42ca7bd52 Added Bang! actor scraper. Fixed date interpolation. Showing date and age of death on profile (only if actor has already died). 2020-05-17 04:59:09 +02:00
ThePendulum 7f86399033 Fixed averaging interpolating function. 2020-05-17 03:04:58 +02:00
ThePendulum 985ab9d2dc Added profile interpolation. 2020-05-17 03:00:44 +02:00
ThePendulum 05ee57378a Storing actor avatars. Using 1 second interval queue for location resolve as per OSM code of conduct. 2020-05-16 04:36:45 +02:00
ThePendulum 21d4dd6bfa Scraping and storing actor profiles. 2020-05-15 04:40:59 +02:00
ThePendulum 11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
ThePendulum f1eb29c713 Added thumb and lazy image scripts. Added FreeOnes and Boobpedia as sites. 2020-05-13 23:17:39 +02:00
ThePendulum dac451bb86 Using alias actor ID when available. Using basic SVG flags instead of PNG. Moved PNG and HD SVG flags to assets. 2020-05-13 20:27:06 +02:00
ThePendulum 6040a3f41f Associating actors without network. 2020-05-13 02:56:20 +02:00
ThePendulum 525995615a Changed range toggles to links for home and actor pages. 2020-05-07 03:20:51 +02:00
ThePendulum 2b2fb9e3e7 Removed memory usage logs. 2020-05-07 02:16:20 +02:00
ThePendulum c410294022 Added lazy-loading to release tile posters. 2020-05-07 02:10:28 +02:00
ThePendulum af9d8f7858 Drastically improved memory performance media module using streams and temp files. 2020-05-07 01:53:07 +02:00
ThePendulum 79c7847f1c Preserving aspect ratio on release tiles. 2020-04-27 04:00:16 +02:00
ThePendulum f684923a8a Actor tiles maintain aspect ratio. 2020-04-27 02:37:30 +02:00
ThePendulum a223f933ce Added 2 minute timeout to media fetch. 2020-04-26 04:21:57 +02:00
ThePendulum 2ac879d276 Fixed Vixen scraper, using new token URL for trailers. 2020-04-26 03:51:59 +02:00
ThePendulum 2cfbd21560 Adding media streaming experiments. 2020-04-20 23:52:37 +02:00
ThePendulum 4b26f8f476 Added covers to media module. Fixed 'New' sorting for all pages. 2020-04-16 22:54:07 +02:00
ThePendulum 3b2bf921f3 Replaced batch ranges with 'New' sorting. 2020-04-16 22:23:25 +02:00
ThePendulum e3dfaf4440 Staging release media store by role. 2020-04-16 16:24:59 +02:00
ThePendulum fc58850e56 Added media limit sampling. 2020-04-11 22:49:37 +02:00
ThePendulum cb68319ac0 Added lazy loading to tag photos. Changed tag thumb location. 2020-04-08 14:50:43 +02:00
ThePendulum c39bfb234d Blurring video when paused in SFW mode. 2020-04-02 02:23:54 +02:00
ThePendulum ad7874649f Added self hash filtering to media module. Moved Girl Girl back to Jules Jordan. 2020-04-02 01:10:50 +02:00
ThePendulum 9a712e7371 Media module saves files. 2020-03-31 04:05:31 +02:00
ThePendulum 4eaacf5697 Expanded new media module. Added network to channel site to fix actor glitch. 2020-03-30 03:01:08 +02:00
ThePendulum a5d859471d Added Jesse Loads Monster Facials. Added various logos. 2020-03-29 23:42:41 +02:00
ThePendulum 93d4f0ff1a Added WIP media module. Returning releases from release search database function. Fixed page loop in update module. 2020-03-29 04:00:46 +02:00
ThePendulum 6d9f96c5d5 Improved duplicate filtering, now also applying to upcoming updates. Updated Gamma fetchLatest method parameters. Added shortcut for SFW-mode. 2020-03-28 04:37:04 +01:00
ThePendulum 95d115b585 Added Aziani. Added URL origin parameter to relevant qu methods. 2020-03-28 01:40:02 +01:00
ThePendulum 238ebcbf34 Moved SFW mode to server, added HTTP header trigger. 2020-03-27 17:37:13 +01:00
ThePendulum fb59bf552a Added actor assignment to new actors module. Showing network icon on network-specific actors. Improved dark theme. Changed tag tile design. Added Digital Playground logos. 2020-03-27 04:39:13 +01:00
ThePendulum d29e296799 Added database structure for profiles and tattoos. Improved sidebar appearance. Expanded new actors module. 2020-03-26 03:32:07 +01:00
ThePendulum d724f96728 Added new actors module boilerplate. Added tag posters. 2020-03-24 03:48:24 +01:00
ThePendulum 58ead7b426 Added dark and SFW modes. 2020-03-23 01:43:49 +01:00
ThePendulum fdb2b132f6 Improved release storage module. Added new tags module. Added movie scraping. 2020-03-22 03:50:24 +01:00
ThePendulum d765543b30 Improved update runner. Improved HTTP module API, added default user agent. Added PornCZ and Czechav logos. 2020-03-21 02:48:24 +01:00
ThePendulum 4b310e9dfa Added configurable proxy to HTTP module (also used by qu). Added network and site URL to search documents. 2020-03-19 01:54:25 +01:00
ThePendulum e4b269956e Attaching channel site and studio to stored releases. 2020-03-17 00:58:03 +01:00
ThePendulum 0f09fd53eb Refactoring deep scrape. Added tag posters. 2020-03-16 04:10:52 +01:00
ThePendulum c8ebe7892a Refactored update scraper into new module. 2020-03-14 02:56:28 +01:00
ThePendulum 8d484ba728 Added tag photos. Auto-adding BTS tag to BAM Visions scenes. 2020-03-13 21:54:44 +01:00
ThePendulum 01b59f0a9b Changed withReleases boolean to include object in Vixen and Fame Digital scrapers. 2020-03-12 02:21:59 +01:00
ThePendulum 370605554b Added profile scraper with scenes to BAM Visions. Passing 'includes' object instead of withReleases boolean to Gamma. 2020-03-12 02:19:45 +01:00
ThePendulum 25795e2cce Added latest and scene scrapers to BAM Visions. 2020-03-12 00:59:32 +01:00
ThePendulum 37e188a0df Added site aliases. Migrated various scrapers to qu. Added BAM Visions base. 2020-03-12 00:15:25 +01:00
ThePendulum 6a950c8b57 Fixed logger in web server. 2020-03-11 03:03:19 +01:00
ThePendulum 0d0acb6f3c Fixed release sites for profile scraping. 2020-03-11 03:01:37 +01:00
ThePendulum d97b1ab894 Split up profile scrape runner. Fixed wrong search document date key. Added search update CLI. 2020-03-10 23:46:55 +01:00
ThePendulum db63be8f92 Photo plucker will use discarded photos as fallback. Returning high res photo sources from LegalPorno. 2020-03-10 04:42:15 +01:00
ThePendulum 5c55750c0c Fixed qu issues. Fixed media issues. Simplified and expanded date component in search query. 2020-03-10 00:17:57 +01:00
ThePendulum 61a795d634 Added 'includes' argument to scrapers to help them avoid unnecessary requests. Added movie actors and movie tags views. 2020-03-09 16:54:45 +01:00