Commit Graph
273 Commits
Author SHA1 Message Date
pendulum 96eef822d6 Allowing mobile parameter to be set on Gamma network. Added mobile to 21Naturals and 21Sextreme, moved 21Sextury mobile to network. 2020-03-06 19:26:23 +01:00
pendulum 5c63bd860b Tags module filters out undefined tags. Gamma will return [] instead of undefined when no tags are available. 2020-03-06 18:59:32 +01:00
pendulum 90172ea19a Added mobile album support to Gamma scraper. 2020-03-06 04:28:01 +01:00
pendulum fa7b2a7a23 Fixed subnetwork spacing in network sidebar. Using derived entry IDs for main Hush scrapers. 2020-03-06 03:15:49 +01:00
pendulum db01599569 Added profile scraper with releases to Hush. Added qtexts to q to return text nodes individually. Including network in profile site. 2020-03-06 02:49:55 +01:00
pendulum 3889faee26 Added optional sequential scraping and acc release injection. Added Hush Pass and Interracial Pass logos. 2020-03-05 23:01:03 +01:00
pendulum fd6e90e74c Added tour layout scraper to Hush, enabling Interracial POVs, POV Pornstars and See Him Fuck. 2020-03-05 20:31:11 +01:00
pendulum 1407700511 Added Interracial Pass sites. Fixed Hush removing poster from base release. 2020-03-05 17:07:07 +01:00
pendulum 981b121980 Split Hussie Pass into Hush networks Hush Pass, Hussie Pass and Interracial Pass. 2020-03-05 05:07:14 +01:00
pendulum f10e4af29b Allowing scrapers to force channel allocation attempt. Added Hush Pass subsite handling to Hussie Pass scraper. 2020-03-05 03:44:27 +01:00
pendulum 956afa6ae7 Added Hussie Pass scraper. 2020-03-05 02:47:52 +01:00
pendulum 6c3cba1b87 Added actor photos to Brazzers scene scrape. Added no-video poster to Score. Not flattening actor avatar fallbacks. 2020-03-04 17:21:40 +01:00
pendulum 15af3e91e0 Coalescing shoot ID in search. Added stop words for common TLDs. Sorting tags in search results. 2020-03-02 04:15:47 +01:00
pendulum e79a6b33fb Added 'newly added' filter. Handling paywalled videos in Private scraper. Added shoot ID to search. 2020-03-02 03:41:41 +01:00
pendulum e57f440665 Added Amateur Allure. 2020-03-01 05:28:08 +01:00
pendulum 8dd5925af6 Improved search engine query and added stop words. Added 'secondary' property to tag aliases, for tag aliases to be included in searches and alias lists. 2020-02-29 22:47:48 +01:00
pendulum b03775fa07 Using generic slugify for MindGeek channel. 2020-02-29 05:00:50 +01:00
pendulum a828fee476 Handling NULL actors and tags in search table query. Added limit parameter to home URL, default to 30. 2020-02-29 03:22:51 +01:00
pendulum f1f33080f6 Ignoring undefined video entropy. 2020-02-28 03:56:58 +01:00
pendulum 3dc8547431 Added fake data and Markov experiments. 2020-02-27 05:44:24 +01:00
pendulum 97cf5b2b6b Added seed file for test data. 2020-02-27 00:38:11 +01:00
pendulum 3c30e9107a Using dedicated releases search table for ts vector documents. 2020-02-26 22:33:15 +01:00
pendulum 51ffcb5be7 Added opt-out trim to capitalize util. 2020-02-26 04:10:01 +01:00
pendulum 646ff064a7 Updated Score scraper to accept site and with-releases argument. 2020-02-25 22:32:13 +01:00
pendulum 800a25743d Storing image dimensions and file size to database. Added new site Filthy Femdom to Kink. 2020-02-24 04:01:58 +01:00
pendulum 6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
pendulum 0ae7d2669a Added profile scraping to Private. 2020-02-24 00:31:36 +01:00
pendulum 164681427a Fixed Kelly Madison posters, marking frontpage video as teaser. 2020-02-23 22:37:14 +01:00
pendulum 0d719d88ea Removed Boob Pedia as gender source, unreliable. 2020-02-23 22:26:14 +01:00
pendulum 8359f78e2e Fixed RK scraper returning dick size as bust size. 2020-02-23 22:01:12 +01:00
pendulum 0ede7e0f82 Added various Porn Pros archive sites. 2020-02-23 05:00:08 +01:00
pendulum 74274f879e Re-added image entropy filter. 2020-02-22 23:50:25 +01:00
pendulum 915eb75719 Refactored Vixen scraper, using API endpoint and added actor profile and releases scraper. Release scraper will return base release when present and 'deep' argument is false. 2020-02-22 23:25:10 +01:00
pendulum 5e1a1005f1 Refactored Vixen scene scraper. Using better poster source for Vixen. Returning video as teaser instead of trailer. 2020-02-22 05:29:02 +01:00
pendulum e5c6ccd252 Scraping upcoming Vixen scenes. Fetching release media groups sequentially to prevent collisions. 2020-02-22 04:37:48 +01:00
pendulum 349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00
pendulum 13a8221b92 Added New Sensations. Returning null from q's date formatter when date is invalid. 2020-02-21 03:51:57 +01:00
pendulum 7ac5a8e08c Catching media failures per batch. Refined teaser logging. 2020-02-20 22:27:00 +01:00
pendulum 377970f874 Added parent-child relations to network, showing parent network in sidebar. Added Burning Angel using Gamma API. 2020-02-20 02:35:23 +01:00
pendulum 2c490753a4 Removed stray console log from Gamma scraper. 2020-02-19 04:50:10 +01:00
pendulum 8889ea5cf3 Using hasTeaser param instead of date comparison to skip upcoming release teasers in Gamma scraper. 2020-02-19 04:49:54 +01:00
pendulum 372ff54261 Added teaser source to Gamma scraper. Not adding Gamma teaser for upcoming releases. 2020-02-19 04:47:20 +01:00
pendulum 97f5e49187 Refactored media module. Returning 320p and 720p videos from MindGeek as teasers instead of trailers. 2020-02-19 04:41:53 +01:00
pendulum 40bf476ea6 Fixed Porn Pros scraper. Added various Score site logos. 2020-02-18 16:00:36 +01:00
pendulum d1532e1bec Fixed scene URLs for Full Porn Network. 2020-02-15 02:47:50 +01:00
pendulum 30e58fb397 Added actor description to Full Porn Network profile scraper. 2020-02-15 02:06:21 +01:00
pendulum b6fe91b016 Added profile avatar and release scraping to Full Porn Network. 2020-02-15 02:04:46 +01:00
pendulum 880cc64022 Added Full Porn Network scraper. 2020-02-15 01:50:22 +01:00
pendulum 5e461f1f1a Added proper Gamma parameters to 21Sextury sites. 2020-02-13 23:05:28 +01:00
pendulum 1ff8d37d89 Added upcoming support to Nubiles. Renamed q's formatDate to extractDate, added actual formatDate. 2020-02-13 04:11:32 +01:00