Commit Graph

558 Commits

Author SHA1 Message Date
8dd5925af6 Improved search engine query and added stop words. Added 'secondary' property to tag aliases, for tag aliases to be included in searches and alias lists. 2020-02-29 22:47:48 +01:00
b03775fa07 Using generic slugify for MindGeek channel. 2020-02-29 05:00:50 +01:00
a828fee476 Handling NULL actors and tags in search table query. Added limit parameter to home URL, default to 30. 2020-02-29 03:22:51 +01:00
f1f33080f6 Ignoring undefined video entropy. 2020-02-28 03:56:58 +01:00
3dc8547431 Added fake data and Markov experiments. 2020-02-27 05:44:24 +01:00
97cf5b2b6b Added seed file for test data. 2020-02-27 00:38:11 +01:00
3c30e9107a Using dedicated releases search table for ts vector documents. 2020-02-26 22:33:15 +01:00
51ffcb5be7 Added opt-out trim to capitalize util. 2020-02-26 04:10:01 +01:00
646ff064a7 Updated Score scraper to accept site and with-releases argument. 2020-02-25 22:32:13 +01:00
800a25743d Storing image dimensions and file size to database. Added new site Filthy Femdom to Kink. 2020-02-24 04:01:58 +01:00
6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
0ae7d2669a Added profile scraping to Private. 2020-02-24 00:31:36 +01:00
164681427a Fixed Kelly Madison posters, marking frontpage video as teaser. 2020-02-23 22:37:14 +01:00
0d719d88ea Removed Boob Pedia as gender source, unreliable. 2020-02-23 22:26:14 +01:00
8359f78e2e Fixed RK scraper returning dick size as bust size. 2020-02-23 22:01:12 +01:00
0ede7e0f82 Added various Porn Pros archive sites. 2020-02-23 05:00:08 +01:00
74274f879e Re-added image entropy filter. 2020-02-22 23:50:25 +01:00
915eb75719 Refactored Vixen scraper, using API endpoint and added actor profile and releases scraper. Release scraper will return base release when present and 'deep' argument is false. 2020-02-22 23:25:10 +01:00
5e1a1005f1 Refactored Vixen scene scraper. Using better poster source for Vixen. Returning video as teaser instead of trailer. 2020-02-22 05:29:02 +01:00
e5c6ccd252 Scraping upcoming Vixen scenes. Fetching release media groups sequentially to prevent collisions. 2020-02-22 04:37:48 +01:00
349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00
13a8221b92 Added New Sensations. Returning null from q's date formatter when date is invalid. 2020-02-21 03:51:57 +01:00
7ac5a8e08c Catching media failures per batch. Refined teaser logging. 2020-02-20 22:27:00 +01:00
377970f874 Added parent-child relations to network, showing parent network in sidebar. Added Burning Angel using Gamma API. 2020-02-20 02:35:23 +01:00
2c490753a4 Removed stray console log from Gamma scraper. 2020-02-19 04:50:10 +01:00
8889ea5cf3 Using hasTeaser param instead of date comparison to skip upcoming release teasers in Gamma scraper. 2020-02-19 04:49:54 +01:00
372ff54261 Added teaser source to Gamma scraper. Not adding Gamma teaser for upcoming releases. 2020-02-19 04:47:20 +01:00
97f5e49187 Refactored media module. Returning 320p and 720p videos from MindGeek as teasers instead of trailers. 2020-02-19 04:41:53 +01:00
40bf476ea6 Fixed Porn Pros scraper. Added various Score site logos. 2020-02-18 16:00:36 +01:00
d1532e1bec Fixed scene URLs for Full Porn Network. 2020-02-15 02:47:50 +01:00
30e58fb397 Added actor description to Full Porn Network profile scraper. 2020-02-15 02:06:21 +01:00
b6fe91b016 Added profile avatar and release scraping to Full Porn Network. 2020-02-15 02:04:46 +01:00
880cc64022 Added Full Porn Network scraper. 2020-02-15 01:50:22 +01:00
5e461f1f1a Added proper Gamma parameters to 21Sextury sites. 2020-02-13 23:05:28 +01:00
1ff8d37d89 Added upcoming support to Nubiles. Renamed q's formatDate to extractDate, added actual formatDate. 2020-02-13 04:11:32 +01:00
c9c7a33585 Added Cherry Pimps/Pimp.XXX. 2020-02-13 03:44:04 +01:00
b7abd805e2 Added Girl Girl to Jules Jordan. 2020-02-12 23:49:22 +01:00
cd2ca65903 Added Nubiles network. 2020-02-12 23:00:32 +01:00
98d29c0af0 Added Insex fallback for Paintoy. 2020-02-12 16:26:08 +01:00
48b10d0f49 Using brief mode on Insex. 2020-02-12 04:59:15 +01:00
b8074205ef Added Insex. Renamed q's stand-alone date function. Separated q's trim function. Release tile uses cover if available, and poster is not available. 2020-02-12 04:39:57 +01:00
2f70de8e11 Added preflight method to scrapers. Added Assylum. 2020-02-12 01:54:54 +01:00
df4a2b0bb3 Fetching old scene links from Vivid search API. 2020-02-11 19:05:12 +01:00
dd6a1d9bfd Added Vivid network. Added ASMR Fantasy to Adult Time. Storing deep URL in database. Added href to header links. 2020-02-11 04:58:18 +01:00
ce448da7e0 Added 21 Naturals and 21 Sextreme sites. 2020-02-10 23:11:11 +01:00
139f0ce7cb Allowing release scrapers to return actor details. Added True Amateurs. 2020-02-09 23:25:54 +01:00
0f513266a0 Added Black for Wife to JayRock. Switched parameters field to JSON type. 2020-02-09 19:41:39 +01:00
9d9eda29be Added scene count to actor inspect. Preferring network slug over data brand for scene URLs in MindGeek scraper, since milehighmedia.com's brand is milehigh, resulting in milehigh.com. 2020-02-09 03:09:06 +01:00
e61ed2bb5f Added 21Sextreme and 21Naturals networks. Scraping all actor release pages for Brazzers. 2020-02-09 02:01:39 +01:00
8ca458a8ff Added Moms on Moms to Girlsway. 2020-02-08 05:16:04 +01:00