 ThePendulum
|
6950a76cb5
|
Abstracted fragments and curation. Using GraphQL for tags.
|
2019-12-16 05:30:25 +01:00 |
|
ThePendulum
|
f4c2e6c08c
|
Replaced default height and weight fields with fields taking units argument.
|
2019-12-16 02:39:13 +01:00 |
|
ThePendulum
|
577c03f9b7
|
Fixed queries.
|
2019-12-15 23:46:42 +01:00 |
|
ThePendulum
|
ce92d13327
|
Fixed site fallback handling.
|
2019-12-15 23:14:43 +01:00 |
|
ThePendulum
|
07a6c77ce2
|
Improvements, GrapQL experiments. Fixed Babel dependencies.
|
2019-12-15 22:16:55 +01:00 |
|
ThePendulum
|
7ba716cd6f
|
Experimenting using GraphQL in favor of REST.
|
2019-12-15 05:42:51 +01:00 |
|
ThePendulum
|
fed2b0be8a
|
When poster is not available during scraping, first photo is stored as poster; removed client-side fallback. Added screencap album fallback to Jules Jordan scraper. Simplified JJ page traversal.
|
2019-12-13 16:59:04 +01:00 |
|
ThePendulum
|
c998fcf933
|
Added photo album support to Blowpass scraper.
|
2019-12-13 05:04:04 +01:00 |
|
ThePendulum
|
e69b7744c8
|
Showing movie covers as poster when available.
|
2019-12-13 03:57:01 +01:00 |
|
ThePendulum
|
77307d2d13
|
Refactored media module to generalize avatar and poster storage into photo storage.
|
2019-12-13 03:28:52 +01:00 |
|
ThePendulum
|
a03a00f5d9
|
Added Sperm Swallowers and The Ass Factory to Jules Jordan scraper. Added photo source.
|
2019-12-12 05:18:43 +01:00 |
|
ThePendulum
|
0b819713b5
|
Fixed off-by-one in photo plucker. Fixed source duplicate photo function not handling fallback sources.
|
2019-12-12 04:04:35 +01:00 |
|
ThePendulum
|
a310f9bb1d
|
Added support for release photo fallbacks. Limited photo fetching concurrency. Modifier XEmpire scraper for AllBlackX support and improved photo scraping. Added movie property to Evil Angel scraper.
|
2019-12-12 03:12:05 +01:00 |
|
ThePendulum
|
b9af7255f6
|
Fixed release actor origin bug.
|
2019-12-11 05:15:23 +01:00 |
|
ThePendulum
|
18f37e3fbd
|
Moved all tag photos to public asset directory.
|
2019-12-11 02:25:25 +01:00 |
|
ThePendulum
|
b9bac6d8f9
|
Added Kelly Madison profile scraper.
|
2019-12-10 22:35:00 +01:00 |
|
ThePendulum
|
3956994580
|
Swapped poster and photo for Kelly Madison scraper.
|
2019-12-10 05:19:08 +01:00 |
|
ThePendulum
|
85e55eebaf
|
Added Kelly Madison Media (Teen Fidelity) release scraper.
|
2019-12-10 05:10:08 +01:00 |
|
ThePendulum
|
b6123b9418
|
Added source maps. Fixed DDFNetwork avatar and improved photo fetch error handling.
|
2019-12-10 00:30:55 +01:00 |
|
ThePendulum
|
04a89efa58
|
Refactored 21sextury scraper.
|
2019-12-09 05:00:49 +01:00 |
|
ThePendulum
|
222260d3cf
|
Generalized tags to allow them to be assigned to anything.
|
2019-12-07 04:41:16 +01:00 |
|
ThePendulum
|
30f4a418e3
|
Allowing scrapers to return multiple trailer qualities, implemented for Brazzers. Removed tag and site matching from Brazzers scraper.
|
2019-12-07 04:17:14 +01:00 |
|
ThePendulum
|
c4d620c6ff
|
Refactored media module for better duplicate handling, readability and DRY.
|
2019-12-07 03:59:55 +01:00 |
|
ThePendulum
|
98b9c885ec
|
Added support for Reality Kings sites 'Look At Her Now' and 'Tranny Surprise'.
|
2019-12-07 00:42:47 +01:00 |
|
ThePendulum
|
7cb888801f
|
Updated Naughty America scraper, added media support..
|
2019-12-05 04:26:54 +01:00 |
|
ThePendulum
|
2697aaa0af
|
Removed release_date selection, not working as desired. Removed site and tag fetching from Kink scraper.
|
2019-12-05 01:33:57 +01:00 |
|
ThePendulum
|
71cb85c3e1
|
Allowing scrapers to return raw tags and site URLs or slugs, to gradually remove site and tag fetching from individual scrapers. Added media and deep fetchin support to Perv City scraper.
|
2019-12-05 01:26:22 +01:00 |
|
ThePendulum
|
55e3130062
|
Move tag posters and photos to media database.
|
2019-12-04 21:58:08 +01:00 |
|
ThePendulum
|
cf81aa99e0
|
Added rudimentary tags page. Improved social match behavior.
|
2019-12-01 05:32:47 +01:00 |
|
ThePendulum
|
bead69de49
|
Added basic actor and network overview. Added DDF Network actor scraper. Various bug fixes and layout improvements.
|
2019-11-30 05:55:32 +01:00 |
|
ThePendulum
|
0dbe853f39
|
Resolving actor birth and residence place before storage. Layout improvements.
|
2019-11-29 05:46:06 +01:00 |
|
ThePendulum
|
4be508b388
|
Drastic actor page redesign. Storing one avatar per actor, other profile photos as 'photo' role; no longer assuming first photo is avatar.
|
2019-11-28 05:36:22 +01:00 |
|
ThePendulum
|
884ef248e4
|
Fixed actor birthdate and social links scraping.
|
2019-11-27 05:12:51 +01:00 |
|
ThePendulum
|
d113123778
|
Fixed countries seed file. Updated MOFOS scraper. Improved Reality Kings scraper. Limiting photos for XEmpire scraper.
|
2019-11-27 04:58:38 +01:00 |
|
ThePendulum
|
de36ed97e4
|
Added profile scraping in try/catch block. Re-added boobs enhanced info to actor page.
|
2019-11-21 11:43:07 +01:00 |
|
ThePendulum
|
31a8f1cac1
|
Improved profile scrapers. Sorting avatars on actor page by index.
|
2019-11-21 04:24:55 +01:00 |
|
ThePendulum
|
9224b441e2
|
Added Brazzers and Jules Jordan as profile sources. Changed profile structure for proper bust-waist-hip properties and improved stability.
|
2019-11-21 04:05:32 +01:00 |
|
ThePendulum
|
9fcc40dd17
|
Added avatars. Added PornHub and LegalPorno actor profile scrapers.
|
2019-11-20 04:53:36 +01:00 |
|
ThePendulum
|
a13d92b84e
|
Fixed date bug.
|
2019-11-19 04:41:56 +01:00 |
|
ThePendulum
|
3ec7b15886
|
Improved scraping and association behavior.
|
2019-11-19 04:36:15 +01:00 |
|
ThePendulum
|
e8130c3634
|
Scraping actor profiles from FreeOnes.
|
2019-11-17 03:56:45 +01:00 |
|
ThePendulum
|
bd041c528d
|
Fixed ID number/string incompatability in duplicate detection. Expanded Reality Kings scraper to handle older scenes and fix URLs.
|
2019-11-17 00:45:31 +01:00 |
|
ThePendulum
|
2857387441
|
Added bash CLI shorthand. Properly setting parameters in site results. Removed obsolete URL restriction from Reality Kings scraper.
|
2019-11-16 23:37:33 +01:00 |
|
ThePendulum
|
b489c8fc33
|
Major refactor, cleand up site scrape module, fixed and cleaned up release scrape module. Removed old CLI code
|
2019-11-16 03:33:36 +01:00 |
|
ThePendulum
|
5620dfcb65
|
Added filter bar to all pages. Added 'upcoming' marker. Improved date tidbit.
|
2019-11-15 05:10:59 +01:00 |
|
ThePendulum
|
0575dbc7e4
|
Generalized filters bar, added to network page.
|
2019-11-15 02:37:17 +01:00 |
|
ThePendulum
|
1c3f17ec82
|
Added timerange filters. Refactored releases module for more efficient queries.
|
2019-11-15 01:27:58 +01:00 |
|
ThePendulum
|
bf223adc55
|
Redesigned release page. Added 'single site' marker, linking directly to network page. Fixed Dogfart scraper duration.
|
2019-11-14 05:13:38 +01:00 |
|
ThePendulum
|
117d669a69
|
Added Private Black, improved Private scraper to handle their non-private.com sites.
|
2019-11-14 00:00:29 +01:00 |
|
ThePendulum
|
f9f9cc7977
|
Added tag filter to releases query, enabled on homepage.
|
2019-11-13 03:14:24 +01:00 |
|
ThePendulum
|
832e96ced1
|
Improved site and network pages. Fixed various issues.
|
2019-11-12 01:22:20 +01:00 |
|
ThePendulum
|
681f1bca85
|
Added tag page. Added default 'anal' tag to Vixen scraper for Tushy and Tushy Raw.
|
2019-11-11 05:18:28 +01:00 |
|
ThePendulum
|
50816d7b28
|
Generating thumbnails. Added site overview page.
|
2019-11-11 03:20:00 +01:00 |
|
ThePendulum
|
6ed484af76
|
Moved icons into network directories. Improved network logo handling.
|
2019-11-10 04:47:41 +01:00 |
|
ThePendulum
|
a9b91ff76c
|
Added actors page. Added site logos with overview on network page.
|
2019-11-10 04:20:22 +01:00 |
|
ThePendulum
|
82c9bc664a
|
Added network page and sites back-end. Split release tiles into own component.
|
2019-11-09 04:43:58 +01:00 |
|
ThePendulum
|
e9277028e2
|
Added Mike Adriano scraper (True Anal, All Anal, Swallowed, Nympho). Updated README.
|
2019-11-09 01:22:50 +01:00 |
|
ThePendulum
|
0567b1b2ff
|
Refactored seed files to use programmatic upsert instead of SQL duplicate ignore.
|
2019-11-06 02:47:10 +01:00 |
|
ThePendulum
|
f8f6428e36
|
Added media support to Private scraper.
|
2019-11-05 01:41:13 +01:00 |
|
ThePendulum
|
c466d791f5
|
Removed upsert from store releases, observing effects; fixes ? inserted as . Removed query from stored Dogfart URLs.
|
2019-11-05 00:54:47 +01:00 |
|
ThePendulum
|
e4399380d3
|
Fixed photo plucker to prevent duplicate at end.
|
2019-11-04 06:04:43 +01:00 |
|
ThePendulum
|
5745cd33d8
|
Added Dogfart scraper. Added 'date added' property to release page.
|
2019-11-04 05:47:37 +01:00 |
|
ThePendulum
|
2198c7ceb0
|
Added media support to Bang Bros scraper. Added untracked files.
|
2019-10-31 01:53:26 +01:00 |
|
ThePendulum
|
382e40b651
|
Added studio support. Added studios for LegalPorno. Improved media fetch error handling. Fixed DDFNetwork scraper, added media support.
|
2019-10-30 04:45:42 +01:00 |
|
ThePendulum
|
ffe2564bb3
|
Added full photo album to Jules Jordan scraper.
|
2019-10-29 04:49:27 +01:00 |
|
ThePendulum
|
eb50af7b81
|
Added media support to Brazzers scraper.
|
2019-10-29 03:50:39 +01:00 |
|
ThePendulum
|
d5073a73ce
|
Added media support to XEmpire (HardX) scraper.
|
2019-10-29 03:13:56 +01:00 |
|
ThePendulum
|
a4d936523b
|
Added media to LegalPorno scraper.
|
2019-10-29 01:47:16 +01:00 |
|
ThePendulum
|
5f853792b8
|
Added icon component. Stylized filter bar.
|
2019-10-28 02:54:37 +01:00 |
|
ThePendulum
|
f7f5e8a306
|
Deriving transsexual tag from actors in Evil Angel scraper.
|
2019-10-28 02:12:11 +01:00 |
|
ThePendulum
|
0e784a274f
|
Reworked tag matching. Added tag filter groundwork.
|
2019-10-28 00:58:54 +01:00 |
|
ThePendulum
|
2c32ed6549
|
Completed Evil Angel scraper. Added trans tags.
|
2019-09-28 03:26:04 +02:00 |
|
ThePendulum
|
298eabe56e
|
Improved Vixen and XEmpire scrapers. Added media to Blowpass scraper. Improved release fetch code.
|
2019-09-26 03:27:01 +02:00 |
|
ThePendulum
|
96d8cfbcb6
|
Improved media handling, added trailer support. Fetching media from Vixen network frontpages.
|
2019-09-25 04:52:58 +02:00 |
|
ThePendulum
|
b12d74cbb3
|
Fixed Vixen scraper, site restructured. Added Deeper to Vixen network.
|
2019-09-25 02:44:25 +02:00 |
|
ThePendulum
|
c78f7f957f
|
Merge branch 'master' of unknown.name:pendulum/traxxx
|
2019-09-25 01:38:21 +02:00 |
|
ThePendulum
|
05263a97ea
|
Scrolling content container instead of whole page. Improved title source in XEmpire scraper. Fixed scene page breaking when no photos are available.
|
2019-09-25 01:38:06 +02:00 |
 boiii
|
a9f64a82a2
|
Legalporno title scene ID fix
Old regex would fail on some videos
e.g.
Little Sofi Smile plays with two big cocks AF001
Whitney Wright Gets Punished by Two Big Black Cocks AB015
|
2019-09-23 18:32:47 +02:00 |
|
ThePendulum
|
cf4987ba48
|
Using media database for images.
|
2019-09-10 16:48:04 +02:00 |
|
ThePendulum
|
02e81a8be9
|
Refactored site accumulation to work with new database structure
|
2019-09-08 04:06:49 +02:00 |
|
ThePendulum
|
926c227b85
|
Refactored database to separate IDs from slugs
|
2019-09-08 03:53:09 +02:00 |
|
ThePendulum
|
f2a6b58370
|
Added release page with photo viewer. Added photo count to database. Improved styling.
|
2019-07-06 05:29:12 +02:00 |
|
ThePendulum
|
b8aa81b3f1
|
Transitioning to Vue. Installed environment and restored homepage scene overview.
|
2019-06-03 05:31:38 +02:00 |
|
ThePendulum
|
fda6d1e475
|
Added release page.
|
2019-05-19 01:34:08 +02:00 |
|
ThePendulum
|
e3558fc0c5
|
Using grid layout with thumbnails.
|
2019-05-08 05:50:13 +02:00 |
|
ThePendulum
|
8eb2dcfd89
|
Storing and fetching tags.
|
2019-05-06 23:41:38 +02:00 |
|
ThePendulum
|
3df5a821b6
|
Saving results to database. Showing webpage.
|
2019-05-06 02:01:57 +02:00 |
|
ThePendulum
|
d044aff9dd
|
Destroying knex when done to ensure process exit.
|
2019-04-12 03:54:34 +02:00 |
|
ThePendulum
|
b486b6abdd
|
Added Bang Bros scraper.
|
2019-04-12 03:12:53 +02:00 |
|
ThePendulum
|
82419be402
|
Changed scene fetch title source for BlowPass scraper.
|
2019-04-10 04:44:36 +02:00 |
|
ThePendulum
|
7dd2432837
|
Changed title source for 21Sextury scene fetch,
|
2019-04-10 03:57:34 +02:00 |
|
ThePendulum
|
bc01cbe6dc
|
Added Bang Bros data (no scraper yet). Added 21Sextury scene fetch date fallback.
|
2019-04-10 03:42:20 +02:00 |
|
ThePendulum
|
be74f84781
|
Expanded README.
|
2019-04-08 04:07:39 +02:00 |
|
ThePendulum
|
db05bcf666
|
Added site stats to README.
|
2019-04-08 03:09:37 +02:00 |
|
ThePendulum
|
763ffb341a
|
Added Naughty America scraper.
|
2019-04-08 03:07:16 +02:00 |
|
ThePendulum
|
d526e4daea
|
Added Reality Kings scraper. Improved site finder.
|
2019-04-08 01:49:45 +02:00 |
|
ThePendulum
|
ec056a177a
|
Added 21Sextury scraper. Various improvements.
|
2019-04-07 20:51:14 +02:00 |
|
ThePendulum
|
d5b9fec0db
|
Added duration property to scraper template.
|
2019-04-07 05:21:27 +02:00 |
|
ThePendulum
|
6b9d99f495
|
Added description property to recent scrapers (apparently forgotten). Reconfigured eslint for scrapers.
|
2019-04-07 05:19:44 +02:00 |
|
ThePendulum
|
c8abb302e4
|
Added DDFNetwork scraper. Modified tag matching query to be case insensitive.
|
2019-04-07 05:01:06 +02:00 |
|
ThePendulum
|
885a815022
|
Added tag groups. Added MOFOS scraped. Improved entry ID handling.
|
2019-04-07 02:15:57 +02:00 |
|
ThePendulum
|
3a90f98d41
|
Added Blowpass scraper. Split shootId and pageId.
|
2019-04-06 23:24:26 +02:00 |
|
ThePendulum
|
2b818e379a
|
Scrapers can now iterate through pages. Filtering unique releases before saving to database. Improved scrapers and rendering.
|
2019-04-05 03:45:40 +02:00 |
|
ThePendulum
|
cbb4fdc919
|
Added utils directory to repository.
|
2019-04-04 21:10:44 +02:00 |
|
ThePendulum
|
6190247ac5
|
Added Brazzers scraper. Removed scene-only sites in database in favor of automatic network fallback.
|
2019-04-04 20:22:47 +02:00 |
|
ThePendulum
|
784542253b
|
Expanded tag database.
|
2019-04-04 04:44:37 +02:00 |
|
ThePendulum
|
b3beeef3e4
|
Added Private scraper. Added Vixen scraper to repository.
|
2019-04-04 04:00:28 +02:00 |
|
ThePendulum
|
439d3225ec
|
Added Vixen scraper. Added LegalPorno studio IDs.
|
2019-04-01 02:45:15 +02:00 |
|
ThePendulum
|
8421cd8648
|
Added parameters column to sites database, fixes Perv City scraper. Getting shoot ID from all existing scrapers.
|
2019-03-26 01:26:47 +01:00 |
|
ThePendulum
|
c60ceeabfa
|
Fixed actor list and title trimming for Jules Jordan scraper.
|
2019-03-25 04:15:45 +01:00 |
|
ThePendulum
|
4d7f323e86
|
Replaced network and tag files with SQLite database.
|
2019-03-25 03:57:33 +01:00 |
|
ThePendulum
|
845f46e9ad
|
Added Kink scraper with elaborate site specification.
|
2019-03-24 05:28:18 +01:00 |
|
ThePendulum
|
e85a0f45f9
|
Improved copy & quit options.
|
2019-03-24 03:08:51 +01:00 |
|
ThePendulum
|
4fcabb4aae
|
Added tags and duration to scraping. Added LegalPorno scraper.
|
2019-03-24 01:29:22 +01:00 |
|
ThePendulum
|
e8d4b76403
|
Improved module structure. Added individual scene scrapers for Jules Jordan and XEmpire.
|
2019-03-23 22:48:39 +01:00 |
|
ThePendulum
|
1a170bfbd5
|
Added Jules Jordan scraper. Added render argument to stop going into ncurses mode during dev.
|
2019-03-18 04:46:53 +01:00 |
|
ThePendulum
|
ddbed3c142
|
Added colors to ratings. TUI renderer accounts for style encodings in formatted values.
|
2019-03-11 04:51:28 +01:00 |
|
ThePendulum
|
02c421b737
|
Displaying results in terminal table with select, scroll and experimental search. Expanded broken character handling for PervCity scraper.
|
2019-03-11 04:19:36 +01:00 |
|
ThePendulum
|
af9565b296
|
Completed Perv City scraper. Outputting results as list.
|
2019-03-04 04:19:03 +01:00 |
|
ThePendulum
|
53ea694894
|
Added include and exclude config options. Moved network definitions to dedicated file. Scraping all XEmpire sites. Added definitions for Perv City.
|
2019-03-04 02:46:33 +01:00 |
|
ThePendulum
|
71aa31dda6
|
Experimentally scraping data from HardX.
|
2019-03-03 04:18:33 +01:00 |