DebaucheryLibrarian
|
a3c18ca577
|
Rewrote Jay Rock scraper for new website.
|
2020-11-04 04:02:51 +01:00 |
|
DebaucheryLibrarian
|
8586817963
|
Fixed Insex html table selector.
|
2020-11-01 03:25:30 +01:00 |
|
DebaucheryLibrarian
|
c1fffe5cdb
|
Added alternative layout to Insex for updated Topgrl and Sexually Broken sites.
|
2020-11-01 02:58:21 +01:00 |
|
DebaucheryLibrarian
|
3855d96135
|
Removed 'null' from scene tile URL if no slug is available.
|
2020-10-30 17:51:47 +01:00 |
|
DebaucheryLibrarian
|
39f8c037a5
|
Replaced bhttp with patched fork. Improved Jesse Loads Monster Facials scraper reliability (WIP). Added various tag photos.
|
2020-10-30 17:37:10 +01:00 |
|
DebaucheryLibrarian
|
c37d4ad01f
|
Filtering invalid actors from releases before storing.
|
2020-10-29 16:06:20 +01:00 |
|
DebaucheryLibrarian
|
b188bc5744
|
Filtering out empty or unidentified scenes from update scraper, with warning. Improved Jesse Loads Monster Facials reliability.
|
2020-10-29 15:20:59 +01:00 |
|
DebaucheryLibrarian
|
4469376dd2
|
Using temporary table instead of WHERE IN to stack depth error when finding duplicate actors.
|
2020-10-28 03:50:52 +01:00 |
|
DebaucheryLibrarian
|
bf9b334b73
|
Adding scraper config by scraper slug to current 'includes' parameter.
|
2020-10-28 01:36:13 +01:00 |
|
DebaucheryLibrarian
|
1869877178
|
Added release covers table to flush routine.
|
2020-10-27 02:40:30 +01:00 |
|
DebaucheryLibrarian
|
7257776ba8
|
Mapping avatar to media ID in orphan delete.
|
2020-10-27 02:34:06 +01:00 |
|
DebaucheryLibrarian
|
1d3ec96e8d
|
Detecting profile avatars in orphaned media flush.
|
2020-10-27 02:09:52 +01:00 |
|
DebaucheryLibrarian
|
99a7bfeb4c
|
Added traxxx dummy network.
|
2020-10-26 01:42:38 +01:00 |
|
DebaucheryLibrarian
|
0027be65eb
|
Fixed CzechAV to accomodate teasers instead of photos. Fixed error trying to flush non-existent media file.
|
2020-10-25 21:43:36 +01:00 |
|
DebaucheryLibrarian
|
57a44d6643
|
Expanded README.
|
2020-10-25 01:29:25 +02:00 |
|
DebaucheryLibrarian
|
0bd7fca876
|
Added orphaned media flush and batch release flush.
|
2020-10-25 00:52:40 +02:00 |
|
DebaucheryLibrarian
|
6791053c83
|
Fixed entity alias available through wrong type endpoint.
|
2020-10-20 21:04:29 +02:00 |
|
DebaucheryLibrarian
|
9f3c686913
|
Added thumbnail and favicon to entity REST API.
|
2020-10-20 15:37:42 +02:00 |
|
DebaucheryLibrarian
|
ecdd6d8fb0
|
Added logo path to entity API.
|
2020-10-20 15:28:58 +02:00 |
|
DebaucheryLibrarian
|
60eb599416
|
Added alias to entity search query.
|
2020-10-20 00:25:32 +02:00 |
|
DebaucheryLibrarian
|
6b17f9d1f2
|
Allowing entity to be fetched by alias.
|
2020-10-20 00:21:15 +02:00 |
|
DebaucheryLibrarian
|
d795266114
|
Removed no-date-limit as argument due yargs conflict.
|
2020-10-20 00:08:14 +02:00 |
|
DebaucheryLibrarian
|
2f8fca0327
|
Added missing-date as config and argument alias.
|
2020-10-20 00:05:23 +02:00 |
|
DebaucheryLibrarian
|
4a900cbbeb
|
Renamed nullDateLimit to noDateLimit in config.
|
2020-10-20 00:03:22 +02:00 |
|
DebaucheryLibrarian
|
5bdbb5ec62
|
Renamed null-date-limit to no-date-limit, added old as alias.
|
2020-10-20 00:01:29 +02:00 |
|
DebaucheryLibrarian
|
593ce27312
|
Added rudimentary scene and entity scene remove.
|
2020-10-19 02:02:21 +02:00 |
|
DebaucheryLibrarian
|
2536405dba
|
Fixed entity API database query.
|
2020-10-18 00:01:34 +02:00 |
|
DebaucheryLibrarian
|
ca22aedaaa
|
Added rudimentary API documentation to README.
|
2020-10-17 22:54:00 +02:00 |
|
DebaucheryLibrarian
|
e6c52002f0
|
Added tags and entities to REST API..
|
2020-10-16 23:00:03 +02:00 |
|
DebaucheryLibrarian
|
f38233053e
|
Removed debug log.
|
2020-10-15 01:45:13 +02:00 |
|
DebaucheryLibrarian
|
99a4751c20
|
Returning results from new pagination.
|
2020-10-14 03:17:03 +02:00 |
|
DebaucheryLibrarian
|
7c856c267d
|
Added revised next page determination.
|
2020-10-12 04:08:22 +02:00 |
|
DebaucheryLibrarian
|
8aefb8eddb
|
Added and updated tag photos.
|
2020-10-07 03:40:19 +02:00 |
|
DebaucheryLibrarian
|
3f843cc0fc
|
Added version to stats page.
|
2020-09-25 21:21:26 +02:00 |
|
DebaucheryLibrarian
|
566c20ea7e
|
Added various tag photos. Renamed some toy tags.
|
2020-09-21 05:11:24 +02:00 |
|
DebaucheryLibrarian
|
e78bfe4c22
|
Upgraded knex and pg versions.
|
2020-09-19 00:12:15 +02:00 |
|
DebaucheryLibrarian
|
c94e1aaea9
|
Handling missing trailers in Kink scraper.
|
2020-09-18 23:25:15 +02:00 |
|
DebaucheryLibrarian
|
3789ef51f2
|
(Temporarily) removed studio filter from entity query for performance reasons.
|
2020-09-18 22:43:45 +02:00 |
|
DebaucheryLibrarian
|
3c9468b0f1
|
Fixed wrong MindGeek session acquire URL.
|
2020-09-18 03:27:00 +02:00 |
|
DebaucheryLibrarian
|
a4929819df
|
Using channel URL instead of composed URL for session retrieval, should fix Brazzers.
|
2020-09-18 02:54:05 +02:00 |
|
DebaucheryLibrarian
|
a9fa71e455
|
Fixed predata parameter in Assylum scraper.
|
2020-09-17 14:49:45 +02:00 |
|
DebaucheryLibrarian
|
1a8de4fcf6
|
Added mimetype verification option to media source to ensure server returned a plausible file. Added additional fallbacks to Jules Jordan poster scraper for Amateur Allure.
|
2020-09-17 02:30:58 +02:00 |
|
DebaucheryLibrarian
|
0190ee9531
|
Fixed Porn Doe's poster query. Checking style attribute existence in qu before attempting to use it.
|
2020-09-17 00:35:41 +02:00 |
|
DebaucheryLibrarian
|
6fb15fb591
|
Added Dorcel Club with scene and actor scraping. Added count method to qu.
|
2020-09-16 04:55:30 +02:00 |
|
DebaucheryLibrarian
|
a8c525f4fc
|
Hard-coded Pascal White as the male actor for Pascal's Sub Sluts. Added female gender to all Sub Sluts.
|
2020-09-16 02:38:59 +02:00 |
|
DebaucheryLibrarian
|
7c4dd03a8c
|
Ignoring 'Lockdown Submissions' as actor for Pascals Subsluts.
|
2020-09-16 01:45:58 +02:00 |
|
DebaucheryLibrarian
|
7fd7005776
|
Added profile scraper to Pascals Subsluts.
|
2020-09-16 01:42:15 +02:00 |
|
DebaucheryLibrarian
|
bc34c6edb4
|
Added Bang Bros Vault channel.
|
2020-09-14 16:01:27 +02:00 |
|
DebaucheryLibrarian
|
ba7419d3b0
|
Added basic Pascals Subsluts scraper.
|
2020-09-14 02:40:27 +02:00 |
|
DebaucheryLibrarian
|
beeaebbfb7
|
Added CzechAV.
|
2020-09-14 00:53:41 +02:00 |
|
DebaucheryLibrarian
|
b57b0a38f5
|
Fixed infinite pagination when scraping upcoming scenes.
|
2020-09-13 01:41:52 +02:00 |
|
DebaucheryLibrarian
|
eb6337f6fb
|
Fixed upcoming Jules Jordan scene without teaser breaking scraper.
|
2020-09-11 02:36:36 +02:00 |
|
DebaucheryLibrarian
|
0e8024adf1
|
Changed next page determination to ensure --after is followed even if there are no unique releases.
|
2020-09-10 23:49:24 +02:00 |
|
DebaucheryLibrarian
|
bec097f14d
|
Returning status codes from Gamma fetch.
|
2020-09-10 17:54:23 +02:00 |
|
DebaucheryLibrarian
|
7f8704ee2c
|
Using http instead of bhttp for Gamma.
|
2020-09-10 17:41:29 +02:00 |
|
DebaucheryLibrarian
|
5ee5b270ef
|
Added avatars to Gamma latest API.
|
2020-09-10 03:56:09 +02:00 |
|
DebaucheryLibrarian
|
88eeab410b
|
Added filter parameter to Gamma scraper to distinguish Evil Angel exclusives from channel scenes.
|
2020-09-10 03:43:16 +02:00 |
|
DebaucheryLibrarian
|
0d4893b13c
|
Added Evil Angel channels with logos.
|
2020-09-10 03:17:19 +02:00 |
|
DebaucheryLibrarian
|
ac66606135
|
Including networks in filter count. Preserving query between date ranges. Allowing --latest to be used without --last.
|
2020-09-09 03:45:00 +02:00 |
|
DebaucheryLibrarian
|
8aaa88770f
|
Accounting for Windows paths in logger.
|
2020-09-08 15:44:55 +02:00 |
|
DebaucheryLibrarian
|
03ba35d65a
|
Fixed include object. Fixed qu's undefined URL handling.
|
2020-09-08 03:26:34 +02:00 |
|
DebaucheryLibrarian
|
e90bb63a8f
|
Added American Pornstar. Improved Jules Jordan scraper to accomodate for American Pornstar. Changed entity logo mogrify settings to ensure both minimum height and width.
|
2020-09-05 01:56:54 +02:00 |
|
DebaucheryLibrarian
|
3c84a814a8
|
Added Zero Tolerance with Addicted 2 Girls and GenderX.
|
2020-09-04 03:07:28 +02:00 |
|
DebaucheryLibrarian
|
c187a27123
|
Added Hookup Hotshot.
|
2020-09-03 22:22:12 +02:00 |
|
DebaucheryLibrarian
|
f6353ca14c
|
Storing actor profile URL when provided from scene page.
|
2020-08-31 02:43:41 +02:00 |
|
DebaucheryLibrarian
|
1bfdf4b232
|
Storing actor profiles from scene pages.
|
2020-08-30 04:18:47 +02:00 |
|
DebaucheryLibrarian
|
53b0101a12
|
Added new Naughty America sites.
|
2020-08-26 23:59:29 +02:00 |
|
DebaucheryLibrarian
|
8611d738b0
|
Using UTC to query date ranges. Removed stray console log from MindGeek scraper.
|
2020-08-26 02:01:38 +02:00 |
|
DebaucheryLibrarian
|
52f66e7982
|
Fixed undefined location in FreeOnes scraper.
|
2020-08-24 18:24:07 +02:00 |
|
DebaucheryLibrarian
|
7fed5b7138
|
Moved Brazzers to MindGeek scraper to support new site.
|
2020-08-24 05:13:34 +02:00 |
|
DebaucheryLibrarian
|
d8b8dfa299
|
Not creating batch ID when no movies are to be stored.
|
2020-08-23 03:41:29 +02:00 |
|
DebaucheryLibrarian
|
c3d771c8fc
|
Hush scraper uses children from entity argument for filter regexp, instead of making its own database request.
|
2020-08-23 03:31:37 +02:00 |
|
DebaucheryLibrarian
|
278b74e78c
|
Providing duplicate releases in predata. Using duplicates for filtering scenes without channel in Hush scraper.
|
2020-08-23 02:43:10 +02:00 |
|
DebaucheryLibrarian
|
3a5ea3dd9a
|
Returning duplicate releases from pagination.
|
2020-08-22 04:22:56 +02:00 |
|
DebaucheryLibrarian
|
5f5c48ea05
|
Fixed pagination behavior for upcoming scenes.
|
2020-08-22 01:57:23 +02:00 |
|
DebaucheryLibrarian
|
7c052fedfc
|
Fixed map error.
|
2020-08-22 00:28:22 +02:00 |
|
DebaucheryLibrarian
|
1c17cd1be5
|
Improved update scrape pagination and limits.
|
2020-08-21 03:55:51 +02:00 |
|
DebaucheryLibrarian
|
4ec89e2cc8
|
Added upcoming, profile and detailed scene actor scraping to InTheCrack. Fixed clip upsert.
|
2020-08-20 23:35:18 +02:00 |
|
DebaucheryLibrarian
|
501e764c21
|
Renamed chapters to clips. Fixed Vixen trailers.
|
2020-08-20 19:52:02 +02:00 |
|
DebaucheryLibrarian
|
2835c66694
|
Added chapters and shoot location. Added In The Crack.
|
2020-08-20 04:57:38 +02:00 |
|
DebaucheryLibrarian
|
fd4477bc50
|
Improved 'new' sorting.
|
2020-08-19 21:48:55 +02:00 |
|
DebaucheryLibrarian
|
e896d52968
|
Added separate task queue for video streams to prevent ffmpeg overstressing the CPU. Fixed entity parent in scene REST API.
|
2020-08-17 15:53:20 +02:00 |
|
DebaucheryLibrarian
|
b3435c97c3
|
Added footer and basic stats page.
|
2020-08-15 19:04:33 +02:00 |
|
DebaucheryLibrarian
|
d7974f057f
|
Fixed scene tile detail word wrap. Fixed Score posters.
|
2020-08-15 02:05:35 +02:00 |
|
DebaucheryLibrarian
|
50c5f921f5
|
Using new bulk insert utility for releases, media and actors.
|
2020-08-14 23:21:53 +02:00 |
|
DebaucheryLibrarian
|
e996a45bf5
|
Added new bulk upsert utility.
|
2020-08-14 23:05:25 +02:00 |
|
DebaucheryLibrarian
|
b3f784686f
|
Improved entity provision behavior.
|
2020-08-14 00:32:59 +02:00 |
|
DebaucheryLibrarian
|
77566eae0d
|
Fixed and documented entity configuration and query.
|
2020-08-13 23:59:54 +02:00 |
|
DebaucheryLibrarian
|
59e2124407
|
Removed type property from scenes API.
|
2020-08-13 16:10:58 +02:00 |
|
DebaucheryLibrarian
|
f8c9b69f4b
|
Allowing --actors-update to be used without --actors.
|
2020-08-12 21:04:38 +02:00 |
|
DebaucheryLibrarian
|
d14034d38b
|
Defauling --actors-update to 1900-01-01
|
2020-08-12 21:00:50 +02:00 |
|
DebaucheryLibrarian
|
7413d7db25
|
Improved and documented actor profile scraping.
|
2020-08-12 20:51:08 +02:00 |
|
DebaucheryLibrarian
|
5cabeed19d
|
Modularized release component between movie and scene. Added Kink Classics channel.
|
2020-08-12 03:30:20 +02:00 |
|
DebaucheryLibrarian
|
40aed1086f
|
Showing actors, tags and date on movie tiles.
|
2020-08-10 21:39:55 +02:00 |
|
DebaucheryLibrarian
|
a7d5bef93f
|
Filtering undefined scenes property from movies. Added movie page scraper to Elegant Angel.
|
2020-08-08 18:10:59 +02:00 |
|
DebaucheryLibrarian
|
b4f0501765
|
Scraping from Cherry Pimps when available. Showing cover in movie tile.
|
2020-08-02 03:44:14 +02:00 |
|
DebaucheryLibrarian
|
767437d9aa
|
Added movie tile. Fixed actor header. Larger breakpoint for nav menu.
|
2020-08-01 15:11:07 +02:00 |
|
DebaucheryLibrarian
|
6c5a62353c
|
Moved movies to separate table.
|
2020-07-25 03:44:19 +02:00 |
|
DebaucheryLibrarian
|
bfd54e94e7
|
Improved Team Skeet profile scraper.
|
2020-07-23 21:00:16 +02:00 |
|
DebaucheryLibrarian
|
747c2e1637
|
Checking nationality against alpha2 and alpha2. Improved Team Skeet profile scraper.
|
2020-07-23 04:39:12 +02:00 |