Commit Graph

52 Commits

Author SHA1 Message Date
DebaucheryLibrarian
0633197793 Removed direct bhttp usage from scrapers in favor of local http module. Deleted legacy scrapers, as old code is available via git repo history. 2020-11-23 00:05:02 +01:00
DebaucheryLibrarian
b9b777c621 Using new HTTP module with a dynamic rate limiter. 2020-11-22 04:07:09 +01:00
DebaucheryLibrarian
39f8c037a5 Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos. 2020-10-30 17:37:10 +01:00
DebaucheryLibrarian
1a8de4fcf6 Added mimetype verification option to media source to ensure server returned a plausible file. Added additional fallbacks to Jules Jordan poster scraper for Amateur Allure. 2020-09-17 02:30:58 +02:00
DebaucheryLibrarian
eb6337f6fb Fixed upcoming Jules Jordan scene without teaser breaking scraper. 2020-09-11 02:36:36 +02:00
DebaucheryLibrarian
0e8024adf1 Changed next page determination to ensure --after is followed even if there are no unique releases. 2020-09-10 23:49:24 +02:00
DebaucheryLibrarian
e90bb63a8f Added American Pornstar. Improved Jules Jordan scraper to accomodate for American Pornstar. Changed entity logo mogrify settings to ensure both minimum height and width. 2020-09-05 01:56:54 +02:00
DebaucheryLibrarian
5f5c48ea05 Fixed pagination behavior for upcoming scenes. 2020-08-22 01:57:23 +02:00
DebaucheryLibrarian
767437d9aa Added movie tile. Fixed actor header. Larger breakpoint for nav menu. 2020-08-01 15:11:07 +02:00
DebaucheryLibrarian
dff4d15872 Updated profile scrapers to use base actor instead of actor name. Fixes for Reality Kings and Cherry Pimps scrapers. 2020-07-21 01:44:51 +02:00
DebaucheryLibrarian
689c701f34 Various tag photos. 2020-07-18 05:12:32 +02:00
DebaucheryLibrarian
a3d281192d Reordered scraper arguments. Fixed Jules Jordan scraper for Amateur Allure. 2020-07-17 23:27:59 +02:00
2fcd426b49 Added basic pagination to homepage. 2020-05-22 04:32:16 +02:00
f38be7a706 Added components. 2020-05-22 00:55:11 +02:00
0c4628677f Fixed profile location interpolation. Generalizing ethnicity, hair color and eye color. 2020-05-19 01:10:32 +02:00
4826ae8571 Added oil tag. Removed Fame Digital from profile config. 2020-05-18 04:28:38 +02:00
05ee57378a Storing actor avatars. Using 1 second interval queue for location resolve as per OSM code of conduct. 2020-05-16 04:36:45 +02:00
11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
d765543b30 Improved update runner. Improved HTTP module API, added default user agent. Added PornCZ and Czechav logos. 2020-03-21 02:48:24 +01:00
0f09fd53eb Refactoring deep scrape. Added tag posters. 2020-03-16 04:10:52 +01:00
0d0acb6f3c Fixed release sites for profile scraping. 2020-03-11 03:01:37 +01:00
638757b6e4 Added rudimentary movie relations. 2020-03-09 05:06:37 +01:00
6cbb7f9c1e Major API change for 'q', renamed to 'qu', refactored modules. Fixed Gamma URL entry ID regex. 2020-03-09 02:02:29 +01:00
acad99bdfe Changed q get and geta APIs to include status, refactored scrapers. Showing front- and back-cover on movie tiles and release page (fix). Removed icons from main navigation. Returning scenes from Jules Jordan movie scraper. 2020-03-08 04:23:10 +01:00
e57f440665 Added Amateur Allure. 2020-03-01 05:28:08 +01:00
f1f33080f6 Ignoring undefined video entropy. 2020-02-28 03:56:58 +01:00
6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00
377970f874 Added parent-child relations to network, showing parent network in sidebar. Added Burning Angel using Gamma API. 2020-02-20 02:35:23 +01:00
b7abd805e2 Added Girl Girl to Jules Jordan. 2020-02-12 23:49:22 +01:00
a773e0bf54 Added Manuel Ferrara site to Jules Jordan. Refactored Jules Jordan photo scraper for better compatability and quality. 2020-01-08 05:12:14 +01:00
fed2b0be8a When poster is not available during scraping, first photo is stored as poster; removed client-side fallback. Added screencap album fallback to Jules Jordan scraper. Simplified JJ page traversal. 2019-12-13 16:59:04 +01:00
e69b7744c8 Showing movie covers as poster when available. 2019-12-13 03:57:01 +01:00
77307d2d13 Refactored media module to generalize avatar and poster storage into photo storage. 2019-12-13 03:28:52 +01:00
a03a00f5d9 Added Sperm Swallowers and The Ass Factory to Jules Jordan scraper. Added photo source. 2019-12-12 05:18:43 +01:00
0b819713b5 Fixed off-by-one in photo plucker. Fixed source duplicate photo function not handling fallback sources. 2019-12-12 04:04:35 +01:00
a310f9bb1d Added support for release photo fallbacks. Limited photo fetching concurrency. Modifier XEmpire scraper for AllBlackX support and improved photo scraping. Added movie property to Evil Angel scraper. 2019-12-12 03:12:05 +01:00
04a89efa58 Refactored 21sextury scraper. 2019-12-09 05:00:49 +01:00
0dbe853f39 Resolving actor birth and residence place before storage. Layout improvements. 2019-11-29 05:46:06 +01:00
4be508b388 Drastic actor page redesign. Storing one avatar per actor, other profile photos as 'photo' role; no longer assuming first photo is avatar. 2019-11-28 05:36:22 +01:00
9224b441e2 Added Brazzers and Jules Jordan as profile sources. Changed profile structure for proper bust-waist-hip properties and improved stability. 2019-11-21 04:05:32 +01:00
5620dfcb65 Added filter bar to all pages. Added 'upcoming' marker. Improved date tidbit. 2019-11-15 05:10:59 +01:00
ffe2564bb3 Added full photo album to Jules Jordan scraper. 2019-10-29 04:49:27 +01:00
f2a6b58370 Added release page with photo viewer. Added photo count to database. Improved styling. 2019-07-06 05:29:12 +02:00
e3558fc0c5 Using grid layout with thumbnails. 2019-05-08 05:50:13 +02:00
2b818e379a Scrapers can now iterate through pages. Filtering unique releases before saving to database. Improved scrapers and rendering. 2019-04-05 03:45:40 +02:00
8421cd8648 Added parameters column to sites database, fixes Perv City scraper. Getting shoot ID from all existing scrapers. 2019-03-26 01:26:47 +01:00
c60ceeabfa Fixed actor list and title trimming for Jules Jordan scraper. 2019-03-25 04:15:45 +01:00
4d7f323e86 Replaced network and tag files with SQLite database. 2019-03-25 03:57:33 +01:00
4fcabb4aae Added tags and duration to scraping. Added LegalPorno scraper. 2019-03-24 01:29:22 +01:00