boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,352 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

957 Commits

Author SHA1 Message Date
DebaucheryLibrarian e4e0eb23dd Removed unnecessary depth calculation from entity query. 2021-02-02 01:59:51 +01:00
DebaucheryLibrarian 6364912aa8 Added children to deep release entity, removed database dependency from Perfect Gonzo scraper. 2021-02-02 01:51:22 +01:00
DebaucheryLibrarian d5cdfb36a9 Selecting included networks with infinite parent depth to facilitate scraper resolve. 2021-02-02 01:31:12 +01:00
DebaucheryLibrarian 4b5cd50122 Fixed slug lookup in Perfect Gonzo scraper. 2021-02-01 20:49:08 +01:00
DebaucheryLibrarian aade7490f8 Querying infinite parent depth for deep release entities. 2021-02-01 01:45:30 +01:00
DebaucheryLibrarian 97c088cfb4 Added Anal Only and upcoming scraping to Mike Adriano. Fixed profile expand arrow color. 2021-01-30 17:43:33 +01:00
DebaucheryLibrarian bfb5006e95 Added actor scene URL parameter to Gamma scraper to phase out release URL function. 2021-01-30 01:12:42 +01:00
DebaucheryLibrarian 9535ab7953 Regarding layout parameters from parent and grandparent entities. Removed Evil Angel and Fantasy Massage scraper wrappers. 2021-01-30 00:01:40 +01:00
DebaucheryLibrarian 6938e88fbf Fixed some Gamma scene scrapers. 2021-01-29 04:26:45 +01:00
DebaucheryLibrarian b7aaeada45 Improved movie scraping. 2021-01-29 02:38:05 +01:00
DebaucheryLibrarian 4d89256a4c Merging improvements. 2021-01-27 00:21:58 +01:00
DebaucheryLibrarian b506a00e7d Improved scene merging. Improved Porn World/DDF scraper for poster and title redundancy. Fixed SFW poster showing in NSFW mode. 2021-01-25 23:53:56 +01:00
DebaucheryLibrarian b80eca35d8 Fixed Gamma scraper breaking when scene has no movie. Improved missing thumbnail presentation on mobile scene tiles. 2021-01-25 23:24:51 +01:00
DebaucheryLibrarian dc98fcad5a Scraping Gamma movies. Changed movie detail bar position, and scene detail bar mobile spacing. 2021-01-25 23:01:07 +01:00
DebaucheryLibrarian 486dbc5613 Improved Woodman scraper, added profiles. 2021-01-25 00:10:57 +01:00
DebaucheryLibrarian e9cbf5dab2 Added Woodman Casting X and WUNF update and scene scraper. Fixed actor and fallback media arrays being merged in deep scraper. 2021-01-24 23:31:28 +01:00
DebaucheryLibrarian 297f79f6e2 Restored sorting trailers by quality. 2021-01-24 20:43:34 +01:00
DebaucheryLibrarian 4151412156 Fixed Vixen trailer scraping. Using album instead of expand for actor photos. 2021-01-24 17:16:55 +01:00
DebaucheryLibrarian 985b523031 Scraping movie links from Gamma scene page. Removed expand option from scene page media to make way for album button. 2021-01-22 22:55:20 +01:00
DebaucheryLibrarian bd9b795516 Fixed avatars without entropy being discarded. 2021-01-22 15:40:49 +01:00
DebaucheryLibrarian b16cc26024 Added Zero Tolerance channel to network. 2021-01-22 00:26:06 +01:00
DebaucheryLibrarian a95e409366 Fixed Kink deep scrape photos. Fixed favicon ratio in compact scene tile. Hiding scroll buttons on small screens. 2021-01-19 15:58:27 +01:00
DebaucheryLibrarian 48f247a919 Changed scene media grid layout. 2021-01-17 02:07:02 +01:00
DebaucheryLibrarian 251bb9476d Added Gaywire, modified Bang Bros scraper to accomodate. 2021-01-17 01:43:55 +01:00
DebaucheryLibrarian e3ef0a0d69 Added Top Web Models profile scraper. 2021-01-16 04:10:43 +01:00
DebaucheryLibrarian b9e4764516 Fixed Pascals Sub Sluts interpreting metric as imperial height, filtering unlikely in interpolation. Splitting double actor entries in Top Web Models. 2021-01-15 16:14:48 +01:00
DebaucheryLibrarian b8df8e6507 Added Top Web Models update and scene scraper. 2021-01-15 04:04:32 +01:00
DebaucheryLibrarian 451ffdc48b Added Top Web Models directory. 2021-01-14 02:10:35 +01:00
DebaucheryLibrarian 12c3f0d7b9 Added Top Web Models framework. 2021-01-14 01:37:50 +01:00
DebaucheryLibrarian 39d149c728 Added default deep scrape fetch method. Added Karups scene and profile scraper. Added schoolgirl tag photo. 2021-01-13 21:29:05 +01:00
DebaucheryLibrarian 2bae6f693e Removed stray console log. 2021-01-13 16:08:53 +01:00
DebaucheryLibrarian ae0efccb04 Skipping Babel, updated node version. Improved deep scrape array merge. 2021-01-13 16:08:19 +01:00
DebaucheryLibrarian cdd4220cb9 Fixed release actor crash when no actors are present. 2021-01-13 15:43:53 +01:00
DebaucheryLibrarian f95c80f73f Using poster photo for Pinky XXX. 2021-01-13 01:10:29 +01:00
DebaucheryLibrarian 52356f0f31 Added Pinky XXX, changed ethnicity tags. 2021-01-13 00:52:05 +01:00
DebaucheryLibrarian ad73c11cb4 Using date and title for Nubiles entry ID. 2021-01-11 23:31:33 +01:00
DebaucheryLibrarian db4e74fb99 Fixed Nubiles base poster query, handling trailing commas in qu source set. Added profile scene scraper to Dogfart. Added tag photo. 2021-01-11 16:20:01 +01:00
DebaucheryLibrarian e38922f372 Removed redundant sitename from MindGeek session error. 2021-01-05 16:35:49 +01:00
DebaucheryLibrarian e1d6c9e489 Added site name to MindGeek session error. 2021-01-05 16:34:32 +01:00
DebaucheryLibrarian 9ca2ec6dd0 Fixed parent entity relations in seed file. Fixed MindGeek scraper session URL determination. 2021-01-05 16:27:20 +01:00
DebaucheryLibrarian 7bbb2f3557 Added tag filter dialog. 2021-01-03 22:53:51 +01:00
DebaucheryLibrarian 8739ec08cf Fixed qu init selector. Fixed Aziani scene page scope. 2021-01-02 03:20:39 +01:00
DebaucheryLibrarian 236d4fcde7 Fixed scene media lazy image. Fixed Perv City actor scope. 2021-01-01 04:28:10 +01:00
DebaucheryLibrarian ba3b87471e Added scene flush. Added temporary media insert log for integer out of range error. 2020-12-30 04:17:09 +01:00
DebaucheryLibrarian f0a90db912 Only allow actor flush by name for non-entity specific actors. 2020-12-30 03:39:40 +01:00
DebaucheryLibrarian 8aabcd6443 Added actor flush. 2020-12-30 03:19:09 +01:00
DebaucheryLibrarian af67d733ad Added profile flush. 2020-12-30 02:23:43 +01:00
DebaucheryLibrarian 770e5b75a5 Added profile interpolate command line argument. 2020-12-30 00:16:05 +01:00
DebaucheryLibrarian a8f68f4993 Fixed boolean handling in actor profile curation. 2020-12-29 23:44:38 +01:00
DebaucheryLibrarian 361e8f1bd3 Fixed Kelly Madison title regex. 2020-12-29 18:53:55 +01:00
DebaucheryLibrarian 0f052a0631 Fixed actor boolean logic, addressing missing boob, tattoo and piercing info. Removed scroll background, fixed actor photo load event and padding. 2020-12-29 04:20:33 +01:00
DebaucheryLibrarian bc944c2373 Restored Mike Adriano scraper. 2020-12-29 02:05:22 +01:00
DebaucheryLibrarian 8dd10f7e77 Adjusting tooltip arrow position, added open and close events. Fixed search tooltip layout. 2020-12-29 00:42:02 +01:00
DebaucheryLibrarian 6baa6b0802 Fixed fallback create dates in scene tiles. Fixed Mike Adriano entryIds and trailers for Nympho. 2020-12-28 01:29:34 +01:00
DebaucheryLibrarian 77ec2d3747 Using advanced merge library to maximize scraped data. 2020-12-27 23:43:08 +01:00
DebaucheryLibrarian 9c926a1d81 Fixed Mike Adriano dates for scene pages. 2020-12-27 23:36:15 +01:00
DebaucheryLibrarian ded414577f Fixed default actor avatar allocation. Fixed lazy loading in actor photos component. 2020-12-27 22:45:38 +01:00
DebaucheryLibrarian 27e5583849 Using generic session ID variable for to determine consent warning, rather than dedicated property. 2020-12-20 19:49:57 +01:00
DebaucheryLibrarian 5f4039c5d4 Added sharpness and re-added entropy to avatars, ignoring low-entropy photos as main avatar and in profile photo list. 2020-12-20 04:21:28 +01:00
DebaucheryLibrarian bd77d4347d Tied consent warning to session. 2020-12-19 23:01:17 +01:00
DebaucheryLibrarian be4d025505 Fixed trailer path regex for some Hush sites. 2020-12-19 02:35:41 +01:00
DebaucheryLibrarian 67ed249239 Added poster to API scene overview. 2020-12-19 01:03:15 +01:00
DebaucheryLibrarian 0d41fb48dc Removed outdated movie releases from API. Added API endpoint for scene posters. 2020-12-19 00:40:36 +01:00
DebaucheryLibrarian 4ccd8bf07a Fixed missing initial value in scraper reduce, breaking first network (21 Naturals). 2020-12-19 00:19:38 +01:00
DebaucheryLibrarian d67dca60fc Fixed birthdate interpolation and query. 2020-12-17 03:43:09 +01:00
DebaucheryLibrarian cd8e810c35 Fixed various Kelly Madison scraper issues. 2020-12-17 02:05:01 +01:00
DebaucheryLibrarian cc83b832f1 Attach base actor to scenes scraped from profile. 2020-12-16 00:50:58 +01:00
DebaucheryLibrarian 07f0249717 Upgraded pg, knex and postgraphile. Updated nvmrc version to latest node LTS. Fixed Gamma scraper not passing request headers to new http module. 2020-12-14 03:16:47 +01:00
DebaucheryLibrarian 71196688ae Removed console log. 2020-12-05 02:44:39 +01:00
DebaucheryLibrarian f0bec85ef8 Added Fuck'n'Drive and Jizz On Teens latest layouts to First Anal Quest scraper. 2020-12-05 02:24:31 +01:00
DebaucheryLibrarian 2e0fba3de9 Added First Anal Quest and Double View Casting profile scrapers. 2020-12-04 23:53:20 +01:00
DebaucheryLibrarian be1821b9eb Changed --inspect to --report to avoid conflict with Node's own debug tools. 2020-12-02 21:26:55 +01:00
DebaucheryLibrarian 2656e3adb0 Added First Anal Quest and Double View Casting latest and scene scraper. 2020-12-02 03:17:32 +01:00
DebaucheryLibrarian 6f19a8a642 Using ageFromBirth for age on scene date. Using infinite depth and colors for inspect. 2020-12-01 23:00:14 +01:00
DebaucheryLibrarian 9a61d2305c Added fixed actor age. Added male profiles to Littlr Caprice Dreams scraper. Added various tag photos. 2020-11-29 03:59:47 +01:00
DebaucheryLibrarian 71c884fe48 Improved Little Caprice Dreams scraper. 2020-11-28 00:46:30 +01:00
DebaucheryLibrarian 9a183c7ffb Added channel matching to Little Caprice Dreams. 2020-11-27 03:23:12 +01:00
DebaucheryLibrarian 60485751e2 Added support for upcoming scenes to Bang scraper. 2020-11-26 22:07:54 +01:00
DebaucheryLibrarian 4e559f63e3 Including all children of included networks, separated included children into dedicated property. 2020-11-26 04:26:52 +01:00
DebaucheryLibrarian 1b407254a7 Increased hard limit in release API. 2020-11-26 04:01:01 +01:00
DebaucheryLibrarian 0b86def315 Ignoring non-scene page (possible CF or similar protection) in Vixen scraper. 2020-11-26 03:27:21 +01:00
DebaucheryLibrarian 6633ce78d0 Returning empty array instead of null from empty Vixen page. 2020-11-26 03:14:32 +01:00
DebaucheryLibrarian 54df9d0c78 Fixed empty page breaking Vixen scraper. 2020-11-26 03:13:43 +01:00
DebaucheryLibrarian 980efbc93d Added series as channels with logos and photo album scraping to Little Caprice. Added various tag photos. 2020-11-24 04:29:44 +01:00
DebaucheryLibrarian 711a9441a6 Added Little Caprice Dreams scraped (WIP). 2020-11-23 04:32:56 +01:00
DebaucheryLibrarian 8688c28d0f Replaced queueMethod in media sources for new interval/concurrency options. 2020-11-23 00:38:22 +01:00
DebaucheryLibrarian ca7c8f0afd Updated place resolve module to use new HTTP module. 2020-11-23 00:30:20 +01:00
DebaucheryLibrarian 0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian 3d427f7e1d Allowing HTTP rate limits to be set by configuration or argument. 2020-11-22 23:50:24 +01:00
DebaucheryLibrarian 6a5063cf32 Fixed PornCZ scene photos attribute. 2020-11-22 04:13:21 +01:00
DebaucheryLibrarian 081a5a1e8c Updated HTTP call in Gamma scraper. 2020-11-22 04:09:44 +01:00
DebaucheryLibrarian b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian 77f9193669 Updated Dorcel scraper, added movie support. 2020-11-19 02:01:13 +01:00
DebaucheryLibrarian d14ef90136 Scraping actor scenes from Hussie Pass. Adding entity to actor base releases. 2020-11-15 23:50:04 +01:00
DebaucheryLibrarian df9a6eac05 Updated Hussie Pass scraper for new site design. Added cock size to profiles. 2020-11-15 04:33:24 +01:00
DebaucheryLibrarian b3a5d7f379 Fixed JayRock status response, 2020-11-13 01:23:06 +01:00
DebaucheryLibrarian 8a22ff07a6 Merged legacy JayRock scraper into new scraper for CosPimps. 2020-11-13 01:16:17 +01:00
DebaucheryLibrarian 2063d66550 Ignoring placeholder avatar in Jay Rock scraper. 2020-11-10 22:44:15 +01:00
DebaucheryLibrarian a3c18ca577 Rewrote Jay Rock scraper for new website. 2020-11-04 04:02:51 +01:00
DebaucheryLibrarian 8586817963 Fixed Insex html table selector. 2020-11-01 03:25:30 +01:00
DebaucheryLibrarian c1fffe5cdb Added alternative layout to Insex for updated Topgrl and Sexually Broken sites. 2020-11-01 02:58:21 +01:00
DebaucheryLibrarian 3855d96135 Removed 'null' from scene tile URL if no slug is available. 2020-10-30 17:51:47 +01:00
DebaucheryLibrarian 39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian c37d4ad01f Filtering invalid actors from releases before storing. 2020-10-29 16:06:20 +01:00
DebaucheryLibrarian b188bc5744 Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability. 2020-10-29 15:20:59 +01:00
DebaucheryLibrarian 4469376dd2 Using temporary table instead of WHERE IN to stack depth error when finding duplicate actors. 2020-10-28 03:50:52 +01:00
DebaucheryLibrarian bf9b334b73 Adding scraper config by scraper slug to current 'includes' parameter. 2020-10-28 01:36:13 +01:00
DebaucheryLibrarian 1869877178 Added release covers table to flush routine. 2020-10-27 02:40:30 +01:00
DebaucheryLibrarian 7257776ba8 Mapping avatar to media ID in orphan delete. 2020-10-27 02:34:06 +01:00
DebaucheryLibrarian 1d3ec96e8d Detecting profile avatars in orphaned media flush. 2020-10-27 02:09:52 +01:00
DebaucheryLibrarian 99a7bfeb4c Added traxxx dummy network. 2020-10-26 01:42:38 +01:00
DebaucheryLibrarian 0027be65eb Fixed CzechAV to accomodate teasers instead of photos. Fixed error trying to flush non-existent media file. 2020-10-25 21:43:36 +01:00
DebaucheryLibrarian 57a44d6643 Expanded README. 2020-10-25 01:29:25 +02:00
DebaucheryLibrarian 0bd7fca876 Added orphaned media flush and batch release flush. 2020-10-25 00:52:40 +02:00
DebaucheryLibrarian 6791053c83 Fixed entity alias available through wrong type endpoint. 2020-10-20 21:04:29 +02:00
DebaucheryLibrarian 9f3c686913 Added thumbnail and favicon to entity REST API. 2020-10-20 15:37:42 +02:00
DebaucheryLibrarian ecdd6d8fb0 Added logo path to entity API. 2020-10-20 15:28:58 +02:00
DebaucheryLibrarian 60eb599416 Added alias to entity search query. 2020-10-20 00:25:32 +02:00
DebaucheryLibrarian 6b17f9d1f2 Allowing entity to be fetched by alias. 2020-10-20 00:21:15 +02:00
DebaucheryLibrarian d795266114 Removed no-date-limit as argument due yargs conflict. 2020-10-20 00:08:14 +02:00
DebaucheryLibrarian 2f8fca0327 Added missing-date as config and argument alias. 2020-10-20 00:05:23 +02:00
DebaucheryLibrarian 4a900cbbeb Renamed nullDateLimit to noDateLimit in config. 2020-10-20 00:03:22 +02:00
DebaucheryLibrarian 5bdbb5ec62 Renamed null-date-limit to no-date-limit, added old as alias. 2020-10-20 00:01:29 +02:00
DebaucheryLibrarian 593ce27312 Added rudimentary scene and entity scene remove. 2020-10-19 02:02:21 +02:00
DebaucheryLibrarian 2536405dba Fixed entity API database query. 2020-10-18 00:01:34 +02:00
DebaucheryLibrarian ca22aedaaa Added rudimentary API documentation to README. 2020-10-17 22:54:00 +02:00
DebaucheryLibrarian e6c52002f0 Added tags and entities to REST API.. 2020-10-16 23:00:03 +02:00
DebaucheryLibrarian f38233053e Removed debug log. 2020-10-15 01:45:13 +02:00
DebaucheryLibrarian 99a4751c20 Returning results from new pagination. 2020-10-14 03:17:03 +02:00
DebaucheryLibrarian 7c856c267d Added revised next page determination. 2020-10-12 04:08:22 +02:00
DebaucheryLibrarian 8aefb8eddb Added and updated tag photos. 2020-10-07 03:40:19 +02:00
DebaucheryLibrarian 3f843cc0fc Added version to stats page. 2020-09-25 21:21:26 +02:00
DebaucheryLibrarian 566c20ea7e Added various tag photos. Renamed some toy tags. 2020-09-21 05:11:24 +02:00
DebaucheryLibrarian e78bfe4c22 Upgraded knex and pg versions. 2020-09-19 00:12:15 +02:00
DebaucheryLibrarian c94e1aaea9 Handling missing trailers in Kink scraper. 2020-09-18 23:25:15 +02:00
DebaucheryLibrarian 3789ef51f2 (Temporarily) removed studio filter from entity query for performance reasons. 2020-09-18 22:43:45 +02:00
DebaucheryLibrarian 3c9468b0f1 Fixed wrong MindGeek session acquire URL. 2020-09-18 03:27:00 +02:00
DebaucheryLibrarian a4929819df Using channel URL instead of composed URL for session retrieval, should fix Brazzers. 2020-09-18 02:54:05 +02:00
DebaucheryLibrarian a9fa71e455 Fixed predata parameter in Assylum scraper. 2020-09-17 14:49:45 +02:00
DebaucheryLibrarian 1a8de4fcf6 Added mimetype verification option to media source to ensure server returned a plausible file. Added additional fallbacks to Jules Jordan poster scraper for Amateur Allure. 2020-09-17 02:30:58 +02:00
DebaucheryLibrarian 0190ee9531 Fixed Porn Doe's poster query. Checking style attribute existence in qu before attempting to use it. 2020-09-17 00:35:41 +02:00
DebaucheryLibrarian 6fb15fb591 Added Dorcel Club with scene and actor scraping. Added count method to qu. 2020-09-16 04:55:30 +02:00
DebaucheryLibrarian a8c525f4fc Hard-coded Pascal White as the male actor for Pascal's Sub Sluts. Added female gender to all Sub Sluts. 2020-09-16 02:38:59 +02:00
DebaucheryLibrarian 7c4dd03a8c Ignoring 'Lockdown Submissions' as actor for Pascals Subsluts. 2020-09-16 01:45:58 +02:00
DebaucheryLibrarian 7fd7005776 Added profile scraper to Pascals Subsluts. 2020-09-16 01:42:15 +02:00
DebaucheryLibrarian bc34c6edb4 Added Bang Bros Vault channel. 2020-09-14 16:01:27 +02:00
DebaucheryLibrarian ba7419d3b0 Added basic Pascals Subsluts scraper. 2020-09-14 02:40:27 +02:00
DebaucheryLibrarian beeaebbfb7 Added CzechAV. 2020-09-14 00:53:41 +02:00
DebaucheryLibrarian b57b0a38f5 Fixed infinite pagination when scraping upcoming scenes. 2020-09-13 01:41:52 +02:00
DebaucheryLibrarian eb6337f6fb Fixed upcoming Jules Jordan scene without teaser breaking scraper. 2020-09-11 02:36:36 +02:00
DebaucheryLibrarian 0e8024adf1 Changed next page determination to ensure --after is followed even if there are no unique releases. 2020-09-10 23:49:24 +02:00
DebaucheryLibrarian bec097f14d Returning status codes from Gamma fetch. 2020-09-10 17:54:23 +02:00
DebaucheryLibrarian 7f8704ee2c Using http instead of bhttp for Gamma. 2020-09-10 17:41:29 +02:00
DebaucheryLibrarian 5ee5b270ef Added avatars to Gamma latest API. 2020-09-10 03:56:09 +02:00
DebaucheryLibrarian 88eeab410b Added filter parameter to Gamma scraper to distinguish Evil Angel exclusives from channel scenes. 2020-09-10 03:43:16 +02:00
DebaucheryLibrarian 0d4893b13c Added Evil Angel channels with logos. 2020-09-10 03:17:19 +02:00
DebaucheryLibrarian ac66606135 Including networks in filter count. Preserving query between date ranges. Allowing --latest to be used without --last. 2020-09-09 03:45:00 +02:00
DebaucheryLibrarian 8aaa88770f Accounting for Windows paths in logger. 2020-09-08 15:44:55 +02:00
DebaucheryLibrarian 03ba35d65a Fixed include object. Fixed qu's undefined URL handling. 2020-09-08 03:26:34 +02:00
DebaucheryLibrarian e90bb63a8f Added American Pornstar. Improved Jules Jordan scraper to accomodate for American Pornstar. Changed entity logo mogrify settings to ensure both minimum height and width. 2020-09-05 01:56:54 +02:00
DebaucheryLibrarian 3c84a814a8 Added Zero Tolerance with Addicted 2 Girls and GenderX. 2020-09-04 03:07:28 +02:00
DebaucheryLibrarian c187a27123 Added Hookup Hotshot. 2020-09-03 22:22:12 +02:00
DebaucheryLibrarian f6353ca14c Storing actor profile URL when provided from scene page. 2020-08-31 02:43:41 +02:00
DebaucheryLibrarian 1bfdf4b232 Storing actor profiles from scene pages. 2020-08-30 04:18:47 +02:00
DebaucheryLibrarian 53b0101a12 Added new Naughty America sites. 2020-08-26 23:59:29 +02:00
DebaucheryLibrarian 8611d738b0 Using UTC to query date ranges. Removed stray console log from MindGeek scraper. 2020-08-26 02:01:38 +02:00
DebaucheryLibrarian 52f66e7982 Fixed undefined location in FreeOnes scraper. 2020-08-24 18:24:07 +02:00
DebaucheryLibrarian 7fed5b7138 Moved Brazzers to MindGeek scraper to support new site. 2020-08-24 05:13:34 +02:00
DebaucheryLibrarian d8b8dfa299 Not creating batch ID when no movies are to be stored. 2020-08-23 03:41:29 +02:00
DebaucheryLibrarian c3d771c8fc Hush scraper uses children from entity argument for filter regexp, instead of making its own database request. 2020-08-23 03:31:37 +02:00
DebaucheryLibrarian 278b74e78c Providing duplicate releases in predata. Using duplicates for filtering scenes without channel in Hush scraper. 2020-08-23 02:43:10 +02:00
DebaucheryLibrarian 3a5ea3dd9a Returning duplicate releases from pagination. 2020-08-22 04:22:56 +02:00
DebaucheryLibrarian 5f5c48ea05 Fixed pagination behavior for upcoming scenes. 2020-08-22 01:57:23 +02:00
DebaucheryLibrarian 7c052fedfc Fixed map error. 2020-08-22 00:28:22 +02:00
DebaucheryLibrarian 1c17cd1be5 Improved update scrape pagination and limits. 2020-08-21 03:55:51 +02:00
DebaucheryLibrarian 4ec89e2cc8 Added upcoming, profile and detailed scene actor scraping to InTheCrack. Fixed clip upsert. 2020-08-20 23:35:18 +02:00
DebaucheryLibrarian 501e764c21 Renamed chapters to clips. Fixed Vixen trailers. 2020-08-20 19:52:02 +02:00
DebaucheryLibrarian 2835c66694 Added chapters and shoot location. Added In The Crack. 2020-08-20 04:57:38 +02:00
DebaucheryLibrarian fd4477bc50 Improved 'new' sorting. 2020-08-19 21:48:55 +02:00
DebaucheryLibrarian e896d52968 Added separate task queue for video streams to prevent ffmpeg overstressing the CPU. Fixed entity parent in scene REST API. 2020-08-17 15:53:20 +02:00
DebaucheryLibrarian b3435c97c3 Added footer and basic stats page. 2020-08-15 19:04:33 +02:00
DebaucheryLibrarian d7974f057f Fixed scene tile detail word wrap. Fixed Score posters. 2020-08-15 02:05:35 +02:00
DebaucheryLibrarian 50c5f921f5 Using new bulk insert utility for releases, media and actors. 2020-08-14 23:21:53 +02:00
DebaucheryLibrarian e996a45bf5 Added new bulk upsert utility. 2020-08-14 23:05:25 +02:00
DebaucheryLibrarian b3f784686f Improved entity provision behavior. 2020-08-14 00:32:59 +02:00
DebaucheryLibrarian 77566eae0d Fixed and documented entity configuration and query. 2020-08-13 23:59:54 +02:00
DebaucheryLibrarian 59e2124407 Removed type property from scenes API. 2020-08-13 16:10:58 +02:00
DebaucheryLibrarian f8c9b69f4b Allowing --actors-update to be used without --actors. 2020-08-12 21:04:38 +02:00
DebaucheryLibrarian d14034d38b Defauling --actors-update to 1900-01-01 2020-08-12 21:00:50 +02:00
DebaucheryLibrarian 7413d7db25 Improved and documented actor profile scraping. 2020-08-12 20:51:08 +02:00
DebaucheryLibrarian 5cabeed19d Modularized release component between movie and scene. Added Kink Classics channel. 2020-08-12 03:30:20 +02:00
DebaucheryLibrarian 40aed1086f Showing actors, tags and date on movie tiles. 2020-08-10 21:39:55 +02:00
DebaucheryLibrarian a7d5bef93f Filtering undefined scenes property from movies. Added movie page scraper to Elegant Angel. 2020-08-08 18:10:59 +02:00
DebaucheryLibrarian b4f0501765 Scraping from Cherry Pimps when available. Showing cover in movie tile. 2020-08-02 03:44:14 +02:00
DebaucheryLibrarian 767437d9aa Added movie tile. Fixed actor header. Larger breakpoint for nav menu. 2020-08-01 15:11:07 +02:00
DebaucheryLibrarian 6c5a62353c Moved movies to separate table. 2020-07-25 03:44:19 +02:00
DebaucheryLibrarian bfd54e94e7 Improved Team Skeet profile scraper. 2020-07-23 21:00:16 +02:00
DebaucheryLibrarian 747c2e1637 Checking nationality against alpha2 and alpha2. Improved Team Skeet profile scraper. 2020-07-23 04:39:12 +02:00
DebaucheryLibrarian 23e4f87af0 Added profile scraping and Hoby Buchanon to Team Skeet. 2020-07-23 04:29:46 +02:00
DebaucheryLibrarian 9ef5ea8fb6 Added the awkward pagination to PornCZ scraper. 2020-07-23 00:55:55 +02:00
DebaucheryLibrarian 46c6c4dd21 Added PornCZ. 2020-07-22 04:12:20 +02:00
DebaucheryLibrarian 9d89a38490 Added actor page scene thumbnails to Hitzefrei scraper. 2020-07-21 04:10:16 +02:00
DebaucheryLibrarian d56da74168 Added Hitzefrei. Fixed date averaging. 2020-07-21 04:04:07 +02:00
DebaucheryLibrarian dff4d15872 Updated profile scrapers to use base actor instead of actor name. Fixes for Reality Kings and Cherry Pimps scrapers. 2020-07-21 01:44:51 +02:00
DebaucheryLibrarian 939eba8e61 Changed qu's HTML element detection. Passing base actor instead of actorName to profile scrapers. 2020-07-21 01:16:26 +02:00
DebaucheryLibrarian 5291a87587 Added 'clear all' button to tag filter. Tag name never removes tag. Added actor scene scraping to LegalPorno. 2020-07-19 19:40:21 +02:00
DebaucheryLibrarian 689c701f34 Various tag photos. 2020-07-18 05:12:32 +02:00
DebaucheryLibrarian a3d281192d Reordered scraper arguments. Fixed Jules Jordan scraper for Amateur Allure. 2020-07-17 23:27:59 +02:00
DebaucheryLibrarian f59e809713 Added experimental movie page scraping with Elegant Angel. 2020-07-17 04:33:05 +02:00
DebaucheryLibrarian a88c2f0760 Added m3u8 stream support to media module. Added Elegant Angel. Added regex parameter to qu's number method. Various tags. 2020-07-17 03:39:13 +02:00
DebaucheryLibrarian 66d6322c1d Updated GraphQL queries to Datetime. Updated template to latest guideline. 2020-07-16 15:55:03 +02:00
DebaucheryLibrarian 6adfded074 Refactored Mike Adriano scraper. Changed logo and favicon. Added style methods to qu. 2020-07-16 03:47:07 +02:00
DebaucheryLibrarian 5b886b3917 Improved actor extraction for fcuk scraper. Changed 'copyright' to 'credit'. Redused entity page favicon size. 2020-07-15 05:12:29 +02:00
DebaucheryLibrarian c62df2228b Added scraper for FCUK's coed sites. 2020-07-15 04:51:39 +02:00
DebaucheryLibrarian 17b3ba1272 Added partial 'fcuk' (Exploited College Girls) scraper. Added file parameter for actor names and scene URLs. 2020-07-15 03:24:47 +02:00
DebaucheryLibrarian 90fc7a0d9d Fixed Kink profile scraper returning partial matches. Removed parent from BAM Visions and Vogov. 2020-07-14 21:21:27 +02:00
DebaucheryLibrarian 74b15aa8e9 Added 5K Porn and 5K Teens. 2020-07-14 04:36:14 +02:00
DebaucheryLibrarian b7be97fcf9 Added Amateur Euro, For Bondage, Mamacitaz, TransBella and VIP Sex Vault. Refactored Kelly Madison scraper using qu, fixed trailers and improved reliability. 2020-07-14 03:46:31 +02:00
ThePendulum 1eab3be7f6 Added Lets Doe It scraper. Added timestamp matching to qu's duration method. 2020-07-13 03:51:17 +02:00
ThePendulum 6fd2bc2687 Added Kink profile scraper. Fixed --force causing media collisions. 2020-07-13 00:12:01 +02:00
ThePendulum a7707b7b28 Merged DDF Network with Porn World. Fixed and updated DDF/PW scraper. 2020-07-12 22:36:53 +02:00
ThePendulum 70c60e93ac Scraping Perv City updates to network entity. 2020-07-12 05:25:27 +02:00
ThePendulum 9c8cfe3bdb Re-wrote broken Perv City scraper, added profile scraping. 2020-07-12 05:10:23 +02:00
ThePendulum e8c55512e2 Added image processing options to media module. Cropping Killergram avatars. Overwriting images when --force is used. 2020-07-10 03:42:08 +02:00
ThePendulum 4c551cc15f Fixed pagination for Killergram, added pagination to actor profiles, added Killergram Platinum. Added experimental m3u8 stream support. 2020-07-10 02:01:23 +02:00
ThePendulum a013d73400 Added Killergram. 2020-07-09 04:31:27 +02:00
ThePendulum 44a8ced30c Separated actor expand buttons. Refactored Brazzers scraper. Fixed actor releases not included in shallow scrape. Added number query and data-src default to qu img. Updated README. Removed post-install migrate and seed. 2020-07-09 02:00:54 +02:00
ThePendulum 17d46e804e Updated tag photos. 2020-07-08 04:58:12 +02:00
ThePendulum 9e5d3aa139 Added 'visible' property to entities. Added various Teen Core Club logos. 2020-07-08 02:53:46 +02:00
ThePendulum 0fd1edd8fd Added Teen Core Club profile scraper. Added all TCC channels. Added various TCC logos. 2020-07-07 04:37:12 +02:00
ThePendulum abe7de5701 Added scene scraping to Teen Core Club. 2020-07-06 04:13:48 +02:00
ThePendulum f4ef2d4cc2 Auto-shrinking pagination. Various Teen Core Club curations. 2020-07-05 04:10:35 +02:00
ThePendulum c47fae3a1b Setting composed title as page title. Using Teen Core Club title if it isn't shoot ID. 2020-07-03 04:42:20 +02:00
ThePendulum 48a127409e Added date precision. Added Teen Core Club update scraper. 2020-07-03 04:12:56 +02:00
ThePendulum 945c2c45ce Updated tag page layout. Added shoot date property. Showing parent favicon on compact entity page. Re-added 'new' indicator on tile. Added Family Sinner to Mile High Media. Various fixes and improvements. 2020-07-03 01:28:22 +02:00
ThePendulum 749864e922 Added 'independent' property for network-like channels. Changed release tile design. Adding Brazzers timeline events as tags. Added Property Sex to MindGeek. Changed DP, DAP and DVP tag slugs. Changed Porn Pros logo. Added better BAM Visions and Mug Fucked logos. 2020-07-02 04:04:28 +02:00
ThePendulum 1f444e58ce Allowing image sources to specify queue method. Using 5s queue for Whale Member to avoid CDN time-outs. 2020-07-01 04:47:05 +02:00
ThePendulum 53870fda89 Improved release detail bar behavior. 2020-07-01 00:25:27 +02:00
ThePendulum ff384fb734 Fixed search documents to coalesce empty parent entities. 2020-06-30 01:52:17 +02:00
ThePendulum 98c19b560f Updated mindgeek scraper for entities. Various fixes. 2020-06-28 22:29:18 +02:00
ThePendulum 3462d7af2a Entity refactor. Facilitating channels without parent. 2020-06-28 00:15:13 +02:00
ThePendulum af56378ee2 Refactored various modules for entities. Updated and refactored Kink scraper. 2020-06-27 02:57:30 +02:00
ThePendulum 4959dfd14f Refactored deep and store modules to use entities. 2020-06-25 02:26:25 +02:00
ThePendulum f0a89df6ab Refactoring to use entities over sites and networks. 2020-06-17 04:07:24 +02:00
ThePendulum 1907ce1e54 Changed sites from argument query to group by network. 2020-06-15 03:58:35 +02:00
ThePendulum 79465d9634 Added Teen Core Club. Changed network to entity in GraphQL query. 2020-06-08 03:41:12 +02:00
ThePendulum 09d849eb9d Adding networks and sites as entities, 2020-06-04 01:03:02 +02:00
ThePendulum 8abcc7194a Updated Full Porn Network scraper. 2020-05-29 22:43:03 +02:00
ThePendulum 88a88227c4 Applying tag filters to URL. 2020-05-27 01:40:10 +02:00
ThePendulum 86377fec5f Enabled pagination on network page. 2020-05-26 04:11:29 +02:00
ThePendulum 9843023c1f Added Filthy Family and 'legacy' scraper to Bang Bros. Added trans generic avatar. Added pagination support to site actions. 2020-05-24 03:54:29 +02:00
ThePendulum 75d49517b7 Added pagination to actor overview. Lazy loading actor avatars. Reduced hash digest length. 2020-05-23 04:32:50 +02:00
ThePendulum 2fcd426b49 Added basic pagination to homepage. 2020-05-22 04:32:16 +02:00
ThePendulum f38be7a706 Added components. 2020-05-22 00:55:11 +02:00
ThePendulum 532a4b679b Using batch insert for profiles to prevent errors on large inserts. 2020-05-21 03:44:44 +02:00
ThePendulum 703b77897d Added actor scenes to deep fetch sources. 2020-05-20 03:03:42 +02:00
ThePendulum 7275fb10e3 Improved Gamma scraper for (XEmpire) actors. 2020-05-20 03:00:46 +02:00
ThePendulum b1b7cd6d50 Fixed Whale Member posters and photos. 2020-05-20 02:23:45 +02:00
ThePendulum b6691e1991 Added release type distinction to REST API. 2020-05-20 01:38:58 +02:00
ThePendulum 057362d011 Added basic release and actor API. 2020-05-20 01:11:32 +02:00
ThePendulum 6973d39cbd Replacing one or multiple space-like characters with single space in actor description. Using logo thumbnails in descriptions. 2020-05-19 05:06:01 +02:00
ThePendulum c0898b84d6 Showing all unique descriptions on profile with network logo. Fixed Fame Digital scraper. 2020-05-19 04:46:49 +02:00
ThePendulum 9883c3d9c2 Not scraping existing actor profiles unless --force is used. 2020-05-19 02:02:48 +02:00
ThePendulum 0c4628677f Fixed profile location interpolation. Generalizing ethnicity, hair color and eye color. 2020-05-19 01:10:32 +02:00
ThePendulum 4826ae8571 Added oil tag. Removed Fame Digital from profile config. 2020-05-18 04:28:38 +02:00
ThePendulum 003e07491d Added network to profile context when site is available. 2020-05-18 03:34:41 +02:00
ThePendulum 885aa4f627 Passing context object with site or network instead of scraper slug and 'site or network' to all profile scrapers. 2020-05-18 03:22:03 +02:00
ThePendulum 8733fdc657 Improved actor scraping and display. 2020-05-18 01:22:56 +02:00
ThePendulum af5543190a Showing avatar on release actor tile. Fixed social URL validation. 2020-05-17 05:24:46 +02:00
ThePendulum 61965d480c Added birthday to Bang actor scraper. 2020-05-17 05:08:41 +02:00
ThePendulum f42ca7bd52 Added Bang! actor scraper. Fixed date interpolation. Showing date and age of death on profile (only if actor has already died). 2020-05-17 04:59:09 +02:00
ThePendulum 7f86399033 Fixed averaging interpolating function. 2020-05-17 03:04:58 +02:00
ThePendulum 985ab9d2dc Added profile interpolation. 2020-05-17 03:00:44 +02:00
ThePendulum 05ee57378a Storing actor avatars. Using 1 second interval queue for location resolve as per OSM code of conduct. 2020-05-16 04:36:45 +02:00
ThePendulum 21d4dd6bfa Scraping and storing actor profiles. 2020-05-15 04:40:59 +02:00
ThePendulum 11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
ThePendulum f1eb29c713 Added thumb and lazy image scripts. Added FreeOnes and Boobpedia as sites. 2020-05-13 23:17:39 +02:00
ThePendulum dac451bb86 Using alias actor ID when available. Using basic SVG flags instead of PNG. Moved PNG and HD SVG flags to assets. 2020-05-13 20:27:06 +02:00
ThePendulum 6040a3f41f Associating actors without network. 2020-05-13 02:56:20 +02:00
ThePendulum 525995615a Changed range toggles to links for home and actor pages. 2020-05-07 03:20:51 +02:00
ThePendulum 2b2fb9e3e7 Removed memory usage logs. 2020-05-07 02:16:20 +02:00
ThePendulum c410294022 Added lazy-loading to release tile posters. 2020-05-07 02:10:28 +02:00
ThePendulum af9d8f7858 Drastically improved memory performance media module using streams and temp files. 2020-05-07 01:53:07 +02:00
ThePendulum 79c7847f1c Preserving aspect ratio on release tiles. 2020-04-27 04:00:16 +02:00
ThePendulum f684923a8a Actor tiles maintain aspect ratio. 2020-04-27 02:37:30 +02:00
ThePendulum a223f933ce Added 2 minute timeout to media fetch. 2020-04-26 04:21:57 +02:00
ThePendulum 2ac879d276 Fixed Vixen scraper, using new token URL for trailers. 2020-04-26 03:51:59 +02:00
ThePendulum 2cfbd21560 Adding media streaming experiments. 2020-04-20 23:52:37 +02:00
ThePendulum 4b26f8f476 Added covers to media module. Fixed 'New' sorting for all pages. 2020-04-16 22:54:07 +02:00
ThePendulum 3b2bf921f3 Replaced batch ranges with 'New' sorting. 2020-04-16 22:23:25 +02:00
ThePendulum e3dfaf4440 Staging release media store by role. 2020-04-16 16:24:59 +02:00
ThePendulum fc58850e56 Added media limit sampling. 2020-04-11 22:49:37 +02:00
ThePendulum cb68319ac0 Added lazy loading to tag photos. Changed tag thumb location. 2020-04-08 14:50:43 +02:00
ThePendulum c39bfb234d Blurring video when paused in SFW mode. 2020-04-02 02:23:54 +02:00
ThePendulum ad7874649f Added self hash filtering to media module. Moved Girl Girl back to Jules Jordan. 2020-04-02 01:10:50 +02:00
ThePendulum 9a712e7371 Media module saves files. 2020-03-31 04:05:31 +02:00
ThePendulum 4eaacf5697 Expanded new media module. Added network to channel site to fix actor glitch. 2020-03-30 03:01:08 +02:00
ThePendulum a5d859471d Added Jesse Loads Monster Facials. Added various logos. 2020-03-29 23:42:41 +02:00
ThePendulum 93d4f0ff1a Added WIP media module. Returning releases from release search database function. Fixed page loop in update module. 2020-03-29 04:00:46 +02:00
ThePendulum 6d9f96c5d5 Improved duplicate filtering, now also applying to upcoming updates. Updated Gamma fetchLatest method parameters. Added shortcut for SFW-mode. 2020-03-28 04:37:04 +01:00
ThePendulum 95d115b585 Added Aziani. Added URL origin parameter to relevant qu methods. 2020-03-28 01:40:02 +01:00
ThePendulum 238ebcbf34 Moved SFW mode to server, added HTTP header trigger. 2020-03-27 17:37:13 +01:00
ThePendulum fb59bf552a Added actor assignment to new actors module. Showing network icon on network-specific actors. Improved dark theme. Changed tag tile design. Added Digital Playground logos. 2020-03-27 04:39:13 +01:00
ThePendulum d29e296799 Added database structure for profiles and tattoos. Improved sidebar appearance. Expanded new actors module. 2020-03-26 03:32:07 +01:00
ThePendulum d724f96728 Added new actors module boilerplate. Added tag posters. 2020-03-24 03:48:24 +01:00
ThePendulum 58ead7b426 Added dark and SFW modes. 2020-03-23 01:43:49 +01:00
ThePendulum fdb2b132f6 Improved release storage module. Added new tags module. Added movie scraping. 2020-03-22 03:50:24 +01:00
ThePendulum d765543b30 Improved update runner. Improved HTTP module API, added default user agent. Added PornCZ and Czechav logos. 2020-03-21 02:48:24 +01:00
ThePendulum 4b310e9dfa Added configurable proxy to HTTP module (also used by qu). Added network and site URL to search documents. 2020-03-19 01:54:25 +01:00
ThePendulum e4b269956e Attaching channel site and studio to stored releases. 2020-03-17 00:58:03 +01:00
ThePendulum 0f09fd53eb Refactoring deep scrape. Added tag posters. 2020-03-16 04:10:52 +01:00
ThePendulum c8ebe7892a Refactored update scraper into new module. 2020-03-14 02:56:28 +01:00
ThePendulum 8d484ba728 Added tag photos. Auto-adding BTS tag to BAM Visions scenes. 2020-03-13 21:54:44 +01:00
ThePendulum 01b59f0a9b Changed withReleases boolean to include object in Vixen and Fame Digital scrapers. 2020-03-12 02:21:59 +01:00
ThePendulum 370605554b Added profile scraper with scenes to BAM Visions. Passing 'includes' object instead of withReleases boolean to Gamma. 2020-03-12 02:19:45 +01:00
ThePendulum 25795e2cce Added latest and scene scrapers to BAM Visions. 2020-03-12 00:59:32 +01:00
ThePendulum 37e188a0df Added site aliases. Migrated various scrapers to qu. Added BAM Visions base. 2020-03-12 00:15:25 +01:00
ThePendulum 6a950c8b57 Fixed logger in web server. 2020-03-11 03:03:19 +01:00
ThePendulum 0d0acb6f3c Fixed release sites for profile scraping. 2020-03-11 03:01:37 +01:00
ThePendulum d97b1ab894 Split up profile scrape runner. Fixed wrong search document date key. Added search update CLI. 2020-03-10 23:46:55 +01:00
ThePendulum db63be8f92 Photo plucker will use discarded photos as fallback. Returning high res photo sources from LegalPorno. 2020-03-10 04:42:15 +01:00
ThePendulum 5c55750c0c Fixed qu issues. Fixed media issues. Simplified and expanded date component in search query. 2020-03-10 00:17:57 +01:00
ThePendulum 61a795d634 Added 'includes' argument to scrapers to help them avoid unnecessary requests. Added movie actors and movie tags views. 2020-03-09 16:54:45 +01:00
ThePendulum 638757b6e4 Added rudimentary movie relations. 2020-03-09 05:06:37 +01:00
ThePendulum 6cbb7f9c1e Major API change for 'q', renamed to 'qu', refactored modules. Fixed Gamma URL entry ID regex. 2020-03-09 02:02:29 +01:00
ThePendulum acad99bdfe Changed q get and geta APIs to include status, refactored scrapers. Showing front- and back-cover on movie tiles and release page (fix). Removed icons from main navigation. Returning scenes from Jules Jordan movie scraper. 2020-03-08 04:23:10 +01:00
ThePendulum 628c5a2013 Fixed Gamma scene scraper extracting the wrong scene ID from non-standard URLs. 2020-03-07 23:40:38 +01:00
ThePendulum cd12b53071 Added SexyHub. Added favicon. 2020-03-07 04:27:20 +01:00
ThePendulum ff3e956fc7 Added mobile album scraping to Blowpass, improved wrapper. 2020-03-07 02:35:13 +01:00
ThePendulum ea0e37aa49 Fixed Vivid requesting unavailable HTTPS resource, and breaking on non-existing parameters object. 2020-03-07 00:32:04 +01:00
ThePendulum 6820608a09 Gamma won't attempt to fetch photos when 'photos' parameter is false. 2020-03-06 23:50:52 +01:00
ThePendulum aad91fe4ae Added Girls Under Arrest to Adult Time. Added mobile album support to some Girlsway sites. Fixed Moms On Moms link. 2020-03-06 23:40:15 +01:00
ThePendulum a579e3c88f Enabled mobile album scraping for Burning Angel and Fame Digital. 2020-03-06 23:08:55 +01:00
ThePendulum 5d468b58fd Enabled mobile album fetching for Pure Taboo and Fantasy Massage. 2020-03-06 19:44:20 +01:00
ThePendulum 96eef822d6 Allowing mobile parameter to be set on Gamma network. Added mobile to 21Naturals and 21Sextreme, moved 21Sextury mobile to network. 2020-03-06 19:26:23 +01:00
ThePendulum 5c63bd860b Tags module filters out undefined tags. Gamma will return [] instead of undefined when no tags are available. 2020-03-06 18:59:32 +01:00
ThePendulum 90172ea19a Added mobile album support to Gamma scraper. 2020-03-06 04:28:01 +01:00
ThePendulum fa7b2a7a23 Fixed subnetwork spacing in network sidebar. Using derived entry IDs for main Hush scrapers. 2020-03-06 03:15:49 +01:00
ThePendulum db01599569 Added profile scraper with releases to Hush. Added qtexts to q to return text nodes individually. Including network in profile site. 2020-03-06 02:49:55 +01:00
ThePendulum 3889faee26 Added optional sequential scraping and acc release injection. Added Hush Pass and Interracial Pass logos. 2020-03-05 23:01:03 +01:00
ThePendulum fd6e90e74c Added tour layout scraper to Hush, enabling Interracial POVs, POV Pornstars and See Him Fuck. 2020-03-05 20:31:11 +01:00
ThePendulum 1407700511 Added Interracial Pass sites. Fixed Hush removing poster from base release. 2020-03-05 17:07:07 +01:00
ThePendulum 981b121980 Split Hussie Pass into Hush networks Hush Pass, Hussie Pass and Interracial Pass. 2020-03-05 05:07:14 +01:00
ThePendulum f10e4af29b Allowing scrapers to force channel allocation attempt. Added Hush Pass subsite handling to Hussie Pass scraper. 2020-03-05 03:44:27 +01:00
ThePendulum 956afa6ae7 Added Hussie Pass scraper. 2020-03-05 02:47:52 +01:00
ThePendulum 6c3cba1b87 Added actor photos to Brazzers scene scrape. Added no-video poster to Score. Not flattening actor avatar fallbacks. 2020-03-04 17:21:40 +01:00
ThePendulum 15af3e91e0 Coalescing shoot ID in search. Added stop words for common TLDs. Sorting tags in search results. 2020-03-02 04:15:47 +01:00
ThePendulum e79a6b33fb Added 'newly added' filter. Handling paywalled videos in Private scraper. Added shoot ID to search. 2020-03-02 03:41:41 +01:00
ThePendulum e57f440665 Added Amateur Allure. 2020-03-01 05:28:08 +01:00
ThePendulum 8dd5925af6 Improved search engine query and added stop words. Added 'secondary' property to tag aliases, for tag aliases to be included in searches and alias lists. 2020-02-29 22:47:48 +01:00
ThePendulum b03775fa07 Using generic slugify for MindGeek channel. 2020-02-29 05:00:50 +01:00
ThePendulum a828fee476 Handling NULL actors and tags in search table query. Added limit parameter to home URL, default to 30. 2020-02-29 03:22:51 +01:00
ThePendulum f1f33080f6 Ignoring undefined video entropy. 2020-02-28 03:56:58 +01:00
ThePendulum 3dc8547431 Added fake data and Markov experiments. 2020-02-27 05:44:24 +01:00
ThePendulum 97cf5b2b6b Added seed file for test data. 2020-02-27 00:38:11 +01:00
ThePendulum 3c30e9107a Using dedicated releases search table for ts vector documents. 2020-02-26 22:33:15 +01:00
ThePendulum 51ffcb5be7 Added opt-out trim to capitalize util. 2020-02-26 04:10:01 +01:00
ThePendulum 646ff064a7 Updated Score scraper to accept site and with-releases argument. 2020-02-25 22:32:13 +01:00
ThePendulum 800a25743d Storing image dimensions and file size to database. Added new site Filthy Femdom to Kink. 2020-02-24 04:01:58 +01:00
ThePendulum 6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
ThePendulum 0ae7d2669a Added profile scraping to Private. 2020-02-24 00:31:36 +01:00
ThePendulum 164681427a Fixed Kelly Madison posters, marking frontpage video as teaser. 2020-02-23 22:37:14 +01:00
ThePendulum 0d719d88ea Removed Boob Pedia as gender source, unreliable. 2020-02-23 22:26:14 +01:00
ThePendulum 8359f78e2e Fixed RK scraper returning dick size as bust size. 2020-02-23 22:01:12 +01:00
ThePendulum 0ede7e0f82 Added various Porn Pros archive sites. 2020-02-23 05:00:08 +01:00
ThePendulum 74274f879e Re-added image entropy filter. 2020-02-22 23:50:25 +01:00
ThePendulum 915eb75719 Refactored Vixen scraper, using API endpoint and added actor profile and releases scraper. Release scraper will return base release when present and 'deep' argument is false. 2020-02-22 23:25:10 +01:00
ThePendulum 5e1a1005f1 Refactored Vixen scene scraper. Using better poster source for Vixen. Returning video as teaser instead of trailer. 2020-02-22 05:29:02 +01:00
ThePendulum e5c6ccd252 Scraping upcoming Vixen scenes. Fetching release media groups sequentially to prevent collisions. 2020-02-22 04:37:48 +01:00
ThePendulum 349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00
ThePendulum 13a8221b92 Added New Sensations. Returning null from q's date formatter when date is invalid. 2020-02-21 03:51:57 +01:00
ThePendulum 7ac5a8e08c Catching media failures per batch. Refined teaser logging. 2020-02-20 22:27:00 +01:00
ThePendulum 377970f874 Added parent-child relations to network, showing parent network in sidebar. Added Burning Angel using Gamma API. 2020-02-20 02:35:23 +01:00
ThePendulum 2c490753a4 Removed stray console log from Gamma scraper. 2020-02-19 04:50:10 +01:00
ThePendulum 8889ea5cf3 Using hasTeaser param instead of date comparison to skip upcoming release teasers in Gamma scraper. 2020-02-19 04:49:54 +01:00
ThePendulum 372ff54261 Added teaser source to Gamma scraper. Not adding Gamma teaser for upcoming releases. 2020-02-19 04:47:20 +01:00
ThePendulum 97f5e49187 Refactored media module. Returning 320p and 720p videos from MindGeek as teasers instead of trailers. 2020-02-19 04:41:53 +01:00
ThePendulum 40bf476ea6 Fixed Porn Pros scraper. Added various Score site logos. 2020-02-18 16:00:36 +01:00
ThePendulum d1532e1bec Fixed scene URLs for Full Porn Network. 2020-02-15 02:47:50 +01:00
ThePendulum 30e58fb397 Added actor description to Full Porn Network profile scraper. 2020-02-15 02:06:21 +01:00
ThePendulum b6fe91b016 Added profile avatar and release scraping to Full Porn Network. 2020-02-15 02:04:46 +01:00
ThePendulum 880cc64022 Added Full Porn Network scraper. 2020-02-15 01:50:22 +01:00
ThePendulum 5e461f1f1a Added proper Gamma parameters to 21Sextury sites. 2020-02-13 23:05:28 +01:00
ThePendulum 1ff8d37d89 Added upcoming support to Nubiles. Renamed q's formatDate to extractDate, added actual formatDate. 2020-02-13 04:11:32 +01:00
ThePendulum c9c7a33585 Added Cherry Pimps/Pimp.XXX. 2020-02-13 03:44:04 +01:00
ThePendulum b7abd805e2 Added Girl Girl to Jules Jordan. 2020-02-12 23:49:22 +01:00
ThePendulum cd2ca65903 Added Nubiles network. 2020-02-12 23:00:32 +01:00
ThePendulum 98d29c0af0 Added Insex fallback for Paintoy. 2020-02-12 16:26:08 +01:00
ThePendulum 48b10d0f49 Using brief mode on Insex. 2020-02-12 04:59:15 +01:00
ThePendulum b8074205ef Added Insex. Renamed q's stand-alone date function. Separated q's trim function. Release tile uses cover if available, and poster is not available. 2020-02-12 04:39:57 +01:00
ThePendulum 2f70de8e11 Added preflight method to scrapers. Added Assylum. 2020-02-12 01:54:54 +01:00
ThePendulum df4a2b0bb3 Fetching old scene links from Vivid search API. 2020-02-11 19:05:12 +01:00
ThePendulum dd6a1d9bfd Added Vivid network. Added ASMR Fantasy to Adult Time. Storing deep URL in database. Added href to header links. 2020-02-11 04:58:18 +01:00
ThePendulum ce448da7e0 Added 21 Naturals and 21 Sextreme sites. 2020-02-10 23:11:11 +01:00
ThePendulum 139f0ce7cb Allowing release scrapers to return actor details. Added True Amateurs. 2020-02-09 23:25:54 +01:00
ThePendulum 0f513266a0 Added Black for Wife to JayRock. Switched parameters field to JSON type. 2020-02-09 19:41:39 +01:00
ThePendulum 9d9eda29be Added scene count to actor inspect. Preferring network slug over data brand for scene URLs in MindGeek scraper, since milehighmedia.com's brand is milehigh, resulting in milehigh.com. 2020-02-09 03:09:06 +01:00
ThePendulum e61ed2bb5f Added 21Sextreme and 21Naturals networks. Scraping all actor release pages for Brazzers. 2020-02-09 02:01:39 +01:00
ThePendulum 8ca458a8ff Added Moms on Moms to Girlsway. 2020-02-08 05:16:04 +01:00
ThePendulum d2cb74a252 Added Fantasy Massage sites. Improved Private scraper, added movie link. 2020-02-08 04:52:32 +01:00