Commit Graph

821 Commits

Author SHA1 Message Date
DebaucheryLibrarian e88554666c Reduced media concurrency to assess effect on memory. Moved qu context removal to deep scrape runner. Updated movie graphql queries. 2021-12-05 02:54:55 +01:00
DebaucheryLibrarian abd063a578 Removed window map from http module to prevent memory pile-up with update scraping. 2021-12-04 00:32:28 +01:00
DebaucheryLibrarian 1918dd4ad5 Removing query methods from XEmpire release. 2021-12-02 00:45:22 +01:00
DebaucheryLibrarian 61daf5f037 Added debug log for jsdom window closing. 2021-12-01 23:44:25 +01:00
DebaucheryLibrarian 60c151ff6a Reduced deep scrape concurrency to 1. 2021-12-01 23:39:09 +01:00
DebaucheryLibrarian e41f9fa937 Added waitImmediate to deep scrape, reduced concurrency. 2021-12-01 23:30:10 +01:00
DebaucheryLibrarian e29cbc9fea Closing JSDOM window after deep scrape in an attempt to save memory. Reduced deep scrape concurrency to 5. 2021-12-01 17:26:13 +01:00
DebaucheryLibrarian b93a5715cb Updated Jules Jordan scraper for new update page layout. 2021-12-01 00:00:24 +01:00
DebaucheryLibrarian 87094a9498 Replaced cheerio with qu in Gamma scraper. 2021-11-29 02:49:07 +01:00
DebaucheryLibrarian f04eb72891 Added upcoming scraper to PurgatoryX. 2021-11-28 01:20:39 +01:00
DebaucheryLibrarian 9d7183ac69 Added PurgatoryX scraper. 2021-11-27 23:55:16 +01:00
DebaucheryLibrarian 20d0d860d3 Fixed MindGeek scraper trying to acquire session from mindgeek.com 2021-11-22 02:51:52 +01:00
DebaucheryLibrarian 6b4aa64d74 Improved MindGeek scraper session check to prevent crash when network session isn't available yet. 2021-11-22 02:44:03 +01:00
DebaucheryLibrarian e81864ebde Removed verbose http queued output. 2021-11-21 00:47:32 +01:00
DebaucheryLibrarian 0305a22a24 Removed stray console log from media module. 2021-11-21 00:43:35 +01:00
DebaucheryLibrarian 2be3ea9bbb Explicitly adding headers to http module result. 2021-11-21 00:41:12 +01:00
DebaucheryLibrarian 98c103c625 Fixed uuid import. Changed fs rmdir to rm. 2021-11-21 00:31:09 +01:00
DebaucheryLibrarian f0b7678444 Fixed upsert failing on empty insert array due breaking Knex API change. 2021-11-21 00:19:10 +01:00
DebaucheryLibrarian 26539b74a5 Updated dependencies. Added periodic memory logger. 2021-11-20 23:59:48 +01:00
DebaucheryLibrarian 29b8c5e38e Including unextracted scenes in date determination. 2021-10-28 02:10:30 +02:00
DebaucheryLibrarian 0864154a0e Added unextracted property to keep paginating when extracting scenes. 2021-10-28 01:59:53 +02:00
DebaucheryLibrarian a22c4d5679 Added beforeNetwork hook, used by MindGeek. Added Filthy Kings to Gamma. 2021-10-27 17:19:23 +02:00
DebaucheryLibrarian 100a35b4e8 Added before scene fetch method to prevent e.g. unnecessary session requests, moved scraper assignment to entity lookup. Removed channel URL hostname matching.. 2021-10-26 23:42:32 +02:00
DebaucheryLibrarian 6c5d4389fe Not parsing HTML with jsdom when using http module directly to save memory. Added loading ellipsis to release grid pages. 2021-10-25 02:06:24 +02:00
DebaucheryLibrarian 0e4fd12d70 Added more loggers to http module. 2021-10-20 01:46:56 +02:00
DebaucheryLibrarian 972b15e72d Added memory profiling. 2021-10-17 23:56:49 +02:00
DebaucheryLibrarian 49f891ba44 Ignoring 1-second scene duration from MindGeek API. 2021-10-17 19:59:05 +02:00
DebaucheryLibrarian 167df35d37 Fixed tags module not dealing with empty releases or tags, added origin tag column to chapter tag table. 2021-10-17 00:28:13 +02:00
DebaucheryLibrarian b6ad2903f3 Storing original tags. 2021-10-10 00:04:21 +02:00
DebaucheryLibrarian dbff3e9539 Added Brad Montana. Added teaser link to trailer video. 2021-09-28 20:45:22 +02:00
DebaucheryLibrarian 522584711b Added Teen Mega World scraper. 2021-09-17 03:30:49 +02:00
DebaucheryLibrarian a316da3083 Retrieving Spizoo trailers. 2021-09-14 00:42:51 +02:00
DebaucheryLibrarian b164c5dad7 Added Spizoo. 2021-09-13 01:29:39 +02:00
DebaucheryLibrarian d542889827 Added sections and pagination to stash page. 2021-09-12 00:05:45 +02:00
DebaucheryLibrarian 7bd858f96b Generalized Adult Empire subsite scraper, added West Coast Productions. 2021-08-30 01:13:32 +02:00
DebaucheryLibrarian 69a7e8f13d Fixed PornDoe scraper. 2021-08-26 01:14:54 +02:00
DebaucheryLibrarian 23537e0243 Removed Gamma console logs. 2021-08-24 02:56:59 +02:00
DebaucheryLibrarian 245f69a122 Fixed movie bindings. 2021-08-24 02:54:40 +02:00
DebaucheryLibrarian bb384029ac Disabled eager searching except for entities. Updating movie search document on store. 2021-08-23 01:54:22 +02:00
DebaucheryLibrarian 6a8c9d89cb Using paginated full text search for movies, combined actor search and fetch to allow combining search with filters. 2021-08-22 22:25:20 +02:00
DebaucheryLibrarian f00e37490c Finished Cum Louder scraper, updated Vixen scraper. Added tag posters. 2021-08-17 19:25:10 +02:00
DebaucheryLibrarian b08cb46ae5 Removed stray console log. 2021-08-15 13:21:02 +02:00
DebaucheryLibrarian aacfd1b29d Updated object-merge-advanced API use. 2021-08-15 13:16:48 +02:00
DebaucheryLibrarian a848d6991b Added Diabolic and Cum Louder, added content type expect option to media sources to fix Vixen thumbnails. 2021-08-09 10:31:12 +02:00
DebaucheryLibrarian 6847ef690c Added Arch Angel, updated BAM Visions scraper to accomodate Arch Angel (different network, same unidentified CMS). 2021-07-06 00:01:44 +02:00
DebaucheryLibrarian 23b41fc4f3 Fixed Bang scraper. Added Kink affiliate, tag photos. 2021-07-05 00:06:18 +02:00
DebaucheryLibrarian eb7009832a Added rudimentary affiliate banner setup. Separated login and signup disable. Added various tag photos. 2021-06-28 00:05:24 +02:00
DebaucheryLibrarian d1480da076 Added effective date column. Changed warning page theme. 2021-06-19 18:09:58 +02:00
DebaucheryLibrarian ab1329dd67 Updating entity ID for rescraped scenes with network entry IDs enabled. 2021-06-13 16:49:27 +02:00
DebaucheryLibrarian e9a0700742 Improved upcoming update query. 2021-06-06 01:09:32 +02:00
DebaucheryLibrarian 3f473589ad Using bulk insert utility for alert notifications to prevent duplicate errors (fixed). 2021-06-04 03:22:40 +02:00
DebaucheryLibrarian bed329cd8c Using bulk insert utility for alert notifications to prevent duplicate errors. 2021-06-04 03:10:41 +02:00
DebaucheryLibrarian c979173422 Rescraping upcoming scenes. Fixed language and scene deep scraping for Dorcel scraper. 2021-06-02 03:27:32 +02:00
DebaucheryLibrarian 846b860c06 Hiding scene photos and trailers from guests. 2021-05-15 02:51:52 +02:00
DebaucheryLibrarian 3f55b90ab8 Adding alerted scene to stashes. 2021-04-29 01:45:01 +02:00
DebaucheryLibrarian 8bf9fff7dc Triggering notifications for children of alert entities. Showing icons in alert entity search to distinguish networks and channels. 2021-04-26 00:48:31 +02:00
DebaucheryLibrarian eed563e06f Updating video player when switching scene page. 2021-04-25 04:20:38 +02:00
DebaucheryLibrarian fc1c2fc2f3 Added notification clear, improved notification styling. 2021-04-25 03:08:50 +02:00
DebaucheryLibrarian f8a3bf6a64 Updated scene URLs in Vixen scraper. 2021-04-22 19:49:11 +02:00
DebaucheryLibrarian c5e74c33b7 Improved alert notifications. 2021-04-22 19:44:23 +02:00
DebaucheryLibrarian 95f3b1c03a Added rudimentary notifications for set alerts. 2021-04-17 01:10:45 +02:00
DebaucheryLibrarian 7f25846d55 List alerts in profile 2021-04-05 00:48:03 +02:00
DebaucheryLibrarian d36e52d5d1 Added row level security to alert tables. Added alerts to user query. 2021-04-04 22:52:54 +02:00
DebaucheryLibrarian da0cbced15 Added alert dialog. Fixed image rotation EXIT data being discarded. 2021-04-04 21:52:19 +02:00
DebaucheryLibrarian 010da8954b Fixed MYLF scraper failing when channel is missing, fixed profile measurement matching. Added MYLF Selects channel. 2021-03-29 22:47:43 +02:00
DebaucheryLibrarian c386a9098f Generalized Team Skeet scraper, added MYLF network and various Team Skeet partner channels. 2021-03-29 22:22:56 +02:00
DebaucheryLibrarian 3961f83ef6 Fixed animated poster being overwritten as jpeg. 2021-03-26 01:47:40 +01:00
DebaucheryLibrarian 193af9bab5 Fixed session options in http module. 2021-03-23 15:25:21 +01:00
DebaucheryLibrarian c70f500acc Fixed stashed check breaking on empty stashes in PostGraphile plugins. 2021-03-21 04:34:04 +01:00
DebaucheryLibrarian 9ff70e5578 Separated full heart button into component. 2021-03-21 03:46:59 +01:00
DebaucheryLibrarian 348aa91832 Added stash menu to release page, returning stashes from stash API to avoid reloading or local interpolation. 2021-03-21 03:23:58 +01:00
DebaucheryLibrarian eee47111a6 Added delete stash icons and dialog. 2021-03-20 23:03:13 +01:00
DebaucheryLibrarian 07643870cd Updating stash button locally on actor and scene page. 2021-03-20 18:12:06 +01:00
DebaucheryLibrarian e88cf4e3f4 Separated user page stash component. 2021-03-20 02:49:17 +01:00
DebaucheryLibrarian 4bc6ff846d Added public visibility toggle to user page stashes. 2021-03-20 02:03:30 +01:00
DebaucheryLibrarian e301e2184c Fixed undefined user ID in postgres function. Fixed and improved mobile alignment for new and stash icons on scene tile. 2021-03-20 00:12:12 +01:00
DebaucheryLibrarian 731a2792c5 Added favorite stash heart to scene tiles. 2021-03-19 03:27:48 +01:00
DebaucheryLibrarian 862a29bb6e Using thumbnail size instead of original photo size in image tags. 2021-03-18 04:36:04 +01:00
DebaucheryLibrarian 5a2e93e900 Added various tag photos and descriptions. 2021-03-17 05:11:17 +01:00
DebaucheryLibrarian 336b91c872 Refactored http timeout handling. 2021-03-17 02:09:34 +01:00
DebaucheryLibrarian 0d7a03f3e5 Allowing auth to be disabled in config. 2021-03-16 04:12:05 +01:00
DebaucheryLibrarian 1703e9a541 Moved http timeout cancelation before pipeline to prevent large files from getting canceled. 2021-03-16 03:59:36 +01:00
DebaucheryLibrarian 1fb7d384fb Merge branch 'master' into experimental 2021-03-15 04:16:36 +01:00
DebaucheryLibrarian 7c7b38e869 Inserting user ID to PostGraphile if available. 2021-03-15 04:16:32 +01:00
DebaucheryLibrarian a0be8f0aa3 Accounting for missing options in http utility timeout function. 2021-03-15 04:13:09 +01:00
DebaucheryLibrarian cb447da7d0 Added harder timeouts to http utility. Split owner and query database users. 2021-03-15 04:11:14 +01:00
DebaucheryLibrarian 77b40817f2 Added favorites button to actor page. 2021-03-15 03:30:47 +01:00
DebaucheryLibrarian e371e9725a Added stashes with experimental row security policies. Added tag photos. 2021-03-14 04:54:53 +01:00
DebaucheryLibrarian 816529b0ca Added user sign up and login. 2021-03-13 04:26:24 +01:00
DebaucheryLibrarian 00c06778ef Only scraping profile source matching actor entity, changed avatar entropy cut-off from 6 to 5.5. 2021-03-11 04:16:59 +01:00
DebaucheryLibrarian a71752b18b Associating directors separately from actors. Added tag photos. 2021-03-08 01:06:57 +01:00
DebaucheryLibrarian de460f53b1 Changed album close behavior so album can be closed when visiting URL directly. 2021-03-07 16:54:20 +01:00
DebaucheryLibrarian a275d0c855 Smaller tag logo on small displays. Centering tag tile lazy image to prevent load jumps. Including SASS breakpoint variable file automatically. 2021-03-07 16:39:54 +01:00
DebaucheryLibrarian 44bb9b33d9 Added intermittent process report to debug freezes. Removed original images. 2021-03-07 02:09:37 +01:00
DebaucheryLibrarian 17e6f5a5da Storing directors. 2021-03-07 00:01:02 +01:00
DebaucheryLibrarian 254e933740 Added permanent filter for invalid actor associations. 2021-03-04 00:31:31 +01:00
DebaucheryLibrarian 780993eb63 Added debug log for actor associations. 2021-03-03 13:56:50 +01:00
DebaucheryLibrarian c2ec4c15e3 Added basic filter for actor boob size. 2021-03-01 02:41:53 +01:00
DebaucheryLibrarian eca24a7c65 Added WP boob filter to actors page. 2021-02-28 03:38:54 +01:00
DebaucheryLibrarian 8eebcae85d Using direct parent as release actor entity instead of highest parent. 2021-02-27 21:59:33 +01:00