DebaucheryLibrarian
|
9ea317939b
|
Fixed session reuse in Aylo scraper.
|
2023-12-01 02:09:24 +01:00 |
|
DebaucheryLibrarian
|
98bf7ea9dd
|
Updated all dependencies. Updated MindGeek to Aylo.
|
2023-11-30 03:12:47 +01:00 |
|
DebaucheryLibrarian
|
1ed47c3173
|
Updated Radical scraper. Added town property to location resolve tool.
|
2023-08-14 01:54:17 +02:00 |
|
DebaucheryLibrarian
|
5f2c9eb5df
|
Fixed place cache check.
|
2023-08-03 00:31:07 +02:00 |
|
DebaucheryLibrarian
|
c5d81e94e5
|
Updated OSM API path, added redis caching.
|
2023-08-02 23:12:41 +02:00 |
|
DebaucheryLibrarian
|
8c1f1b69ff
|
Splitting Han titles and actors in Model Media scraper.
|
2023-08-02 02:14:41 +02:00 |
|
DebaucheryLibrarian
|
0905847ffa
|
Updated Jules Jordan scraper.
|
2023-07-06 00:14:38 +02:00 |
|
DebaucheryLibrarian
|
205102ff90
|
Added Meiden van Holland and Vurig Vlaanderen.
|
2023-07-01 21:46:44 +02:00 |
|
DebaucheryLibrarian
|
80334843c9
|
Improved puppeteer bypass, enabled for Team Skeet.
|
2023-06-10 02:05:59 +02:00 |
|
DebaucheryLibrarian
|
adda78f0c6
|
Refactored New Sensations scraper.
|
2023-06-05 02:13:36 +02:00 |
|
DebaucheryLibrarian
|
cd98b9d147
|
Changed deep merge so base release photos are only used when no deep photos are available.
|
2022-12-29 23:07:08 +01:00 |
|
DebaucheryLibrarian
|
74214bc060
|
Added puppeteer to http module, refactored Kink scraper to utilize it.
|
2022-12-28 01:34:12 +01:00 |
|
DebaucheryLibrarian
|
6edd62c337
|
Refactored Bang! scraper to match new website, first use of unprint.
|
2022-11-27 04:22:58 +01:00 |
|
DebaucheryLibrarian
|
2943aea4d8
|
Added showcased migration. Added Love Her Films scraper.
|
2022-10-25 23:13:24 +02:00 |
|
DebaucheryLibrarian
|
0fc37e46d2
|
Fixed media module trying to fetch invalid source URLs. Added Accidental Gangbang to Adult Time.
|
2022-09-27 20:09:46 +02:00 |
|
DebaucheryLibrarian
|
3db8b80164
|
Added CF resolver to http module. Using priority lookup in tags seed.
|
2022-08-15 23:51:51 +02:00 |
|
DebaucheryLibrarian
|
527112d5da
|
Refactored Vixen scraper, moved XEmpire to generic Gamma scraper after site update, added BiPhoria.
|
2022-05-15 23:28:56 +02:00 |
|
DebaucheryLibrarian
|
1d84830423
|
Supporting JSON in scenes file. Added Dogfart link finding utility.
|
2022-04-07 16:06:38 +02:00 |
|
DebaucheryLibrarian
|
fffe9080f3
|
Added stand-alone Spizoo sites.
|
2022-02-07 21:58:27 +01:00 |
|
DebaucheryLibrarian
|
dc187a9a3a
|
Added execute method to qu, removed runScripts from Gamma's fetchMovie to observe effect on memory usage.
|
2021-12-19 23:03:44 +01:00 |
|
DebaucheryLibrarian
|
8e4be06675
|
Closing JSDOM window after deep scrapes in an attempt to save memory.
|
2021-12-18 23:22:35 +01:00 |
|
DebaucheryLibrarian
|
abd063a578
|
Removed window map from http module to prevent memory pile-up with update scraping.
|
2021-12-04 00:32:28 +01:00 |
|
DebaucheryLibrarian
|
1918dd4ad5
|
Removing query methods from XEmpire release.
|
2021-12-02 00:45:22 +01:00 |
|
DebaucheryLibrarian
|
e41f9fa937
|
Added waitImmediate to deep scrape, reduced concurrency.
|
2021-12-01 23:30:10 +01:00 |
|
DebaucheryLibrarian
|
e29cbc9fea
|
Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5.
|
2021-12-01 17:26:13 +01:00 |
|
DebaucheryLibrarian
|
87094a9498
|
Replaced cheerio with qu in Gamma scraper.
|
2021-11-29 02:49:07 +01:00 |
|
DebaucheryLibrarian
|
e81864ebde
|
Removed verbose http queued output.
|
2021-11-21 00:47:32 +01:00 |
|
DebaucheryLibrarian
|
2be3ea9bbb
|
Explicitly adding headers to http module result.
|
2021-11-21 00:41:12 +01:00 |
|
DebaucheryLibrarian
|
98c103c625
|
Fixed uuid import. Changed fs rmdir to rm.
|
2021-11-21 00:31:09 +01:00 |
|
DebaucheryLibrarian
|
f0b7678444
|
Fixed upsert failing on empty insert array due breaking Knex API change.
|
2021-11-21 00:19:10 +01:00 |
|
DebaucheryLibrarian
|
26539b74a5
|
Updated dependencies. Added periodic memory logger.
|
2021-11-20 23:59:48 +01:00 |
|
DebaucheryLibrarian
|
6c5d4389fe
|
Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages.
|
2021-10-25 02:06:24 +02:00 |
|
DebaucheryLibrarian
|
0e4fd12d70
|
Added more loggers to http module.
|
2021-10-20 01:46:56 +02:00 |
|
DebaucheryLibrarian
|
522584711b
|
Added Teen Mega World scraper.
|
2021-09-17 03:30:49 +02:00 |
|
DebaucheryLibrarian
|
7bd858f96b
|
Generalized Adult Empire subsite scraper, added West Coast Productions.
|
2021-08-30 01:13:32 +02:00 |
|
DebaucheryLibrarian
|
bed329cd8c
|
Using bulk insert utility for alert notifications to prevent duplicate errors.
|
2021-06-04 03:10:41 +02:00 |
|
DebaucheryLibrarian
|
c979173422
|
Rescraping upcoming scenes. Fixed language and scene deep scraping for Dorcel scraper.
|
2021-06-02 03:27:32 +02:00 |
|
DebaucheryLibrarian
|
c386a9098f
|
Generalized Team Skeet scraper, added MYLF network and various Team Skeet partner channels.
|
2021-03-29 22:22:56 +02:00 |
|
DebaucheryLibrarian
|
193af9bab5
|
Fixed session options in http module.
|
2021-03-23 15:25:21 +01:00 |
|
DebaucheryLibrarian
|
336b91c872
|
Refactored http timeout handling.
|
2021-03-17 02:09:34 +01:00 |
|
DebaucheryLibrarian
|
1703e9a541
|
Moved http timeout cancelation before pipeline to prevent large files from getting canceled.
|
2021-03-16 03:59:36 +01:00 |
|
DebaucheryLibrarian
|
a0be8f0aa3
|
Accounting for missing options in http utility timeout function.
|
2021-03-15 04:13:09 +01:00 |
|
DebaucheryLibrarian
|
cb447da7d0
|
Added harder timeouts to http utility. Split owner and query database users.
|
2021-03-15 04:11:14 +01:00 |
|
DebaucheryLibrarian
|
a275d0c855
|
Smaller tag logo on small displays. Centering tag tile lazy image to prevent load jumps. Including SASS breakpoint variable file automatically.
|
2021-03-07 16:39:54 +01:00 |
|
DebaucheryLibrarian
|
bb20659934
|
Refactored clips into chapters.
|
2021-02-27 00:37:22 +01:00 |
|
DebaucheryLibrarian
|
f310fec869
|
Added S3 experiment.
|
2021-02-20 01:12:44 +01:00 |
|
DebaucheryLibrarian
|
7ff222ce25
|
Passing recursive parameters to all scraper methods. Using throttle parameters in MindGeek scraper, fixed missing slug breaking scene and actor URLs.
|
2021-02-10 03:23:48 +01:00 |
|
DebaucheryLibrarian
|
79b51eca67
|
Added BaDoink profile scraper. Improved convert wrapper.
|
2021-02-03 21:03:35 +01:00 |
|
DebaucheryLibrarian
|
6d93083581
|
Removed superfluous MindGeek scrapers.
|
2021-02-03 00:46:59 +01:00 |
|
DebaucheryLibrarian
|
e3ef0a0d69
|
Added Top Web Models profile scraper.
|
2021-01-16 04:10:43 +01:00 |