boiii
/
traxxx
forked from DebaucheryLibrarian/traxxx
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,307 Commits 8 Branches 0 Tags 4.7 GiB
Commit Graph

503 Commits

Author SHA1 Message Date
DebaucheryLibrarian 7fed5b7138 Moved Brazzers to MindGeek scraper to support new site. 2020-08-24 05:13:34 +02:00
DebaucheryLibrarian c3d771c8fc Hush scraper uses children from entity argument for filter regexp, instead of making its own database request. 2020-08-23 03:31:37 +02:00
DebaucheryLibrarian 3a5ea3dd9a Returning duplicate releases from pagination. 2020-08-22 04:22:56 +02:00
DebaucheryLibrarian 4ec89e2cc8 Added upcoming, profile and detailed scene actor scraping to InTheCrack. Fixed clip upsert. 2020-08-20 23:35:18 +02:00
DebaucheryLibrarian 501e764c21 Renamed chapters to clips. Fixed Vixen trailers. 2020-08-20 19:52:02 +02:00
DebaucheryLibrarian 2835c66694 Added chapters and shoot location. Added In The Crack. 2020-08-20 04:57:38 +02:00
DebaucheryLibrarian e896d52968 Added separate task queue for video streams to prevent ffmpeg overstressing the CPU. Fixed entity parent in scene REST API. 2020-08-17 15:53:20 +02:00
DebaucheryLibrarian e996a45bf5 Added new bulk upsert utility. 2020-08-14 23:05:25 +02:00
DebaucheryLibrarian 5cabeed19d Modularized release component between movie and scene. Added Kink Classics channel. 2020-08-12 03:30:20 +02:00
DebaucheryLibrarian 40aed1086f Showing actors, tags and date on movie tiles. 2020-08-10 21:39:55 +02:00
DebaucheryLibrarian 767437d9aa Added movie tile. Fixed actor header. Larger breakpoint for nav menu. 2020-08-01 15:11:07 +02:00
DebaucheryLibrarian 6c5a62353c Moved movies to separate table. 2020-07-25 03:44:19 +02:00
DebaucheryLibrarian 126fd5c0ff Added various Team Skeet logos. 2020-07-23 18:27:49 +02:00
DebaucheryLibrarian 23e4f87af0 Added profile scraping and Hoby Buchanon to Team Skeet. 2020-07-23 04:29:46 +02:00
DebaucheryLibrarian 46c6c4dd21 Added PornCZ. 2020-07-22 04:12:20 +02:00
DebaucheryLibrarian d56da74168 Added Hitzefrei. Fixed date averaging. 2020-07-21 04:04:07 +02:00
DebaucheryLibrarian 939eba8e61 Changed qu's HTML element detection. Passing base actor instead of actorName to profile scrapers. 2020-07-21 01:16:26 +02:00
DebaucheryLibrarian 0e4c0d8fff Added channel filter. 2020-07-20 04:20:33 +02:00
DebaucheryLibrarian 5291a87587 Added 'clear all' button to tag filter. Tag name never removes tag. Added actor scene scraping to LegalPorno. 2020-07-19 19:40:21 +02:00
DebaucheryLibrarian 2bb511cd99 Added 'match all' tag filter for actors and toggle to 'match any'. 2020-07-19 03:52:36 +02:00
DebaucheryLibrarian 689c701f34 Various tag photos. 2020-07-18 05:12:32 +02:00
DebaucheryLibrarian a3d281192d Reordered scraper arguments. Fixed Jules Jordan scraper for Amateur Allure. 2020-07-17 23:27:59 +02:00
DebaucheryLibrarian a88c2f0760 Added m3u8 stream support to media module. Added Elegant Angel. Added regex parameter to qu's number method. Various tags. 2020-07-17 03:39:13 +02:00
DebaucheryLibrarian 6adfded074 Refactored Mike Adriano scraper. Changed logo and favicon. Added style methods to qu. 2020-07-16 03:47:07 +02:00
DebaucheryLibrarian 5b886b3917 Improved actor extraction for fcuk scraper. Changed 'copyright' to 'credit'. Redused entity page favicon size. 2020-07-15 05:12:29 +02:00
DebaucheryLibrarian c62df2228b Added scraper for FCUK's coed sites. 2020-07-15 04:51:39 +02:00
DebaucheryLibrarian 17b3ba1272 Added partial 'fcuk' (Exploited College Girls) scraper. Added file parameter for actor names and scene URLs. 2020-07-15 03:24:47 +02:00
DebaucheryLibrarian 90fc7a0d9d Fixed Kink profile scraper returning partial matches. Removed parent from BAM Visions and Vogov. 2020-07-14 21:21:27 +02:00
DebaucheryLibrarian 74b15aa8e9 Added 5K Porn and 5K Teens. 2020-07-14 04:36:14 +02:00
DebaucheryLibrarian b7be97fcf9 Added Amateur Euro, For Bondage, Mamacitaz, TransBella and VIP Sex Vault. Refactored Kelly Madison scraper using qu, fixed trailers and improved reliability. 2020-07-14 03:46:31 +02:00
ThePendulum 70aeb4b989 Updated various tag posters. 2020-07-13 04:32:13 +02:00
ThePendulum 1eab3be7f6 Added Lets Doe It scraper. Added timestamp matching to qu's duration method. 2020-07-13 03:51:17 +02:00
ThePendulum a7707b7b28 Merged DDF Network with Porn World. Fixed and updated DDF/PW scraper. 2020-07-12 22:36:53 +02:00
ThePendulum 9c8cfe3bdb Re-wrote broken Perv City scraper, added profile scraping. 2020-07-12 05:10:23 +02:00
ThePendulum 051556936e Added last Killergram logos. 2020-07-12 01:59:16 +02:00
ThePendulum 37d6b747ac Added Killergram logos to repo. Added Hardcore Chain Smokers logo. 2020-07-11 04:47:49 +02:00
ThePendulum 1c5958d1d2 Added various Killergram logos. 2020-07-11 04:37:40 +02:00
ThePendulum 4c551cc15f Fixed pagination for Killergram, added pagination to actor profiles, added Killergram Platinum. Added experimental m3u8 stream support. 2020-07-10 02:01:23 +02:00
ThePendulum a013d73400 Added Killergram. 2020-07-09 04:31:27 +02:00
ThePendulum 44a8ced30c Separated actor expand buttons. Refactored Brazzers scraper. Fixed actor releases not included in shallow scrape. Added number query and data-src default to qu img. Updated README. Removed post-install migrate and seed. 2020-07-09 02:00:54 +02:00
ThePendulum 17d46e804e Updated tag photos. 2020-07-08 04:58:12 +02:00
ThePendulum 9e5d3aa139 Added 'visible' property to entities. Added various Teen Core Club logos. 2020-07-08 02:53:46 +02:00
ThePendulum 0fd1edd8fd Added Teen Core Club profile scraper. Added all TCC channels. Added various TCC logos. 2020-07-07 04:37:12 +02:00
ThePendulum f4ef2d4cc2 Auto-shrinking pagination. Various Teen Core Club curations. 2020-07-05 04:10:35 +02:00
ThePendulum c47fae3a1b Setting composed title as page title. Using Teen Core Club title if it isn't shoot ID. 2020-07-03 04:42:20 +02:00
ThePendulum 48a127409e Added date precision. Added Teen Core Club update scraper. 2020-07-03 04:12:56 +02:00
ThePendulum 945c2c45ce Updated tag page layout. Added shoot date property. Showing parent favicon on compact entity page. Re-added 'new' indicator on tile. Added Family Sinner to Mile High Media. Various fixes and improvements. 2020-07-03 01:28:22 +02:00
ThePendulum 749864e922 Added 'independent' property for network-like channels. Changed release tile design. Adding Brazzers timeline events as tags. Added Property Sex to MindGeek. Changed DP, DAP and DVP tag slugs. Changed Porn Pros logo. Added better BAM Visions and Mug Fucked logos. 2020-07-02 04:04:28 +02:00
ThePendulum 7d31dd8d52 Fixed seed files for stand-alone channel entities. 2020-06-28 00:44:53 +02:00
ThePendulum 3462d7af2a Entity refactor. Facilitating channels without parent. 2020-06-28 00:15:13 +02:00
ThePendulum 0e8b4caac3 Added generic entity page. 2020-06-27 04:50:13 +02:00
ThePendulum af56378ee2 Refactored various modules for entities. Updated and refactored Kink scraper. 2020-06-27 02:57:30 +02:00
ThePendulum 4959dfd14f Refactored deep and store modules to use entities. 2020-06-25 02:26:25 +02:00
ThePendulum 1907ce1e54 Changed sites from argument query to group by network. 2020-06-15 03:58:35 +02:00
ThePendulum 79465d9634 Added Teen Core Club. Changed network to entity in GraphQL query. 2020-06-08 03:41:12 +02:00
ThePendulum 09d849eb9d Adding networks and sites as entities, 2020-06-04 01:03:02 +02:00
ThePendulum 86377fec5f Enabled pagination on network page. 2020-05-26 04:11:29 +02:00
ThePendulum f4c85b7a67 Updated tags and posters. 2020-05-24 05:14:01 +02:00
ThePendulum 9843023c1f Added Filthy Family and 'legacy' scraper to Bang Bros. Added trans generic avatar. Added pagination support to site actions. 2020-05-24 03:54:29 +02:00
ThePendulum 75d49517b7 Added pagination to actor overview. Lazy loading actor avatars. Reduced hash digest length. 2020-05-23 04:32:50 +02:00
ThePendulum 532a4b679b Using batch insert for profiles to prevent errors on large inserts. 2020-05-21 03:44:44 +02:00
ThePendulum 7275fb10e3 Improved Gamma scraper for (XEmpire) actors. 2020-05-20 03:00:46 +02:00
ThePendulum c0898b84d6 Showing all unique descriptions on profile with network logo. Fixed Fame Digital scraper. 2020-05-19 04:46:49 +02:00
ThePendulum 0c4628677f Fixed profile location interpolation. Generalizing ethnicity, hair color and eye color. 2020-05-19 01:10:32 +02:00
ThePendulum 4826ae8571 Added oil tag. Removed Fame Digital from profile config. 2020-05-18 04:28:38 +02:00
ThePendulum 885aa4f627 Passing context object with site or network instead of scraper slug and 'site or network' to all profile scrapers. 2020-05-18 03:22:03 +02:00
ThePendulum 8733fdc657 Improved actor scraping and display. 2020-05-18 01:22:56 +02:00
ThePendulum 05ee57378a Storing actor avatars. Using 1 second interval queue for location resolve as per OSM code of conduct. 2020-05-16 04:36:45 +02:00
ThePendulum 11eb66f834 Switched to tabs. Adding missing actor entries when scraping actors, with batch ID. 2020-05-14 04:26:05 +02:00
ThePendulum f1eb29c713 Added thumb and lazy image scripts. Added FreeOnes and Boobpedia as sites. 2020-05-13 23:17:39 +02:00
ThePendulum dac451bb86 Using alias actor ID when available. Using basic SVG flags instead of PNG. Moved PNG and HD SVG flags to assets. 2020-05-13 20:27:06 +02:00
ThePendulum 6040a3f41f Associating actors without network. 2020-05-13 02:56:20 +02:00
ThePendulum af9d8f7858 Drastically improved memory performance media module using streams and temp files. 2020-05-07 01:53:07 +02:00
ThePendulum 3b2bf921f3 Replaced batch ranges with 'New' sorting. 2020-04-16 22:23:25 +02:00
ThePendulum cb68319ac0 Added lazy loading to tag photos. Changed tag thumb location. 2020-04-08 14:50:43 +02:00
ThePendulum 24b297011e Added various tag posters. 2020-04-02 04:05:55 +02:00
ThePendulum ad7874649f Added self hash filtering to media module. Moved Girl Girl back to Jules Jordan. 2020-04-02 01:10:50 +02:00
ThePendulum a5d859471d Added Jesse Loads Monster Facials. Added various logos. 2020-03-29 23:42:41 +02:00
ThePendulum 93d4f0ff1a Added WIP media module. Returning releases from release search database function. Fixed page loop in update module. 2020-03-29 04:00:46 +02:00
ThePendulum 6d9f96c5d5 Improved duplicate filtering, now also applying to upcoming updates. Updated Gamma fetchLatest method parameters. Added shortcut for SFW-mode. 2020-03-28 04:37:04 +01:00
ThePendulum 95d115b585 Added Aziani. Added URL origin parameter to relevant qu methods. 2020-03-28 01:40:02 +01:00
ThePendulum 238ebcbf34 Moved SFW mode to server, added HTTP header trigger. 2020-03-27 17:37:13 +01:00
ThePendulum d29e296799 Added database structure for profiles and tattoos. Improved sidebar appearance. Expanded new actors module. 2020-03-26 03:32:07 +01:00
ThePendulum bb3f6fc408 Added more SFW photos. 2020-03-25 04:25:10 +01:00
ThePendulum 15a386ad05 Added compact sidebar. Added tag sections and posters. 2020-03-25 02:48:54 +01:00
ThePendulum d724f96728 Added new actors module boilerplate. Added tag posters. 2020-03-24 03:48:24 +01:00
ThePendulum aa488cc6e3 Changed tags on tag overview. 2020-03-23 04:58:23 +01:00
ThePendulum 58ead7b426 Added dark and SFW modes. 2020-03-23 01:43:49 +01:00
ThePendulum fdb2b132f6 Improved release storage module. Added new tags module. Added movie scraping. 2020-03-22 03:50:24 +01:00
ThePendulum 0f09fd53eb Refactoring deep scrape. Added tag posters. 2020-03-16 04:10:52 +01:00
ThePendulum c8ebe7892a Refactored update scraper into new module. 2020-03-14 02:56:28 +01:00
ThePendulum 8d484ba728 Added tag photos. Auto-adding BTS tag to BAM Visions scenes. 2020-03-13 21:54:44 +01:00
ThePendulum 8e4573e5ad Retouched BAM Visions logo. 2020-03-12 02:35:14 +01:00
ThePendulum 25795e2cce Added latest and scene scrapers to BAM Visions. 2020-03-12 00:59:32 +01:00
ThePendulum 37e188a0df Added site aliases. Migrated various scrapers to qu. Added BAM Visions base. 2020-03-12 00:15:25 +01:00
ThePendulum 61a795d634 Added 'includes' argument to scrapers to help them avoid unnecessary requests. Added movie actors and movie tags views. 2020-03-09 16:54:45 +01:00
ThePendulum 6cbb7f9c1e Major API change for 'q', renamed to 'qu', refactored modules. Fixed Gamma URL entry ID regex. 2020-03-09 02:02:29 +01:00
ThePendulum cd12b53071 Added SexyHub. Added favicon. 2020-03-07 04:27:20 +01:00
ThePendulum ff3e956fc7 Added mobile album scraping to Blowpass, improved wrapper. 2020-03-07 02:35:13 +01:00
ThePendulum ea0e37aa49 Fixed Vivid requesting unavailable HTTPS resource, and breaking on non-existing parameters object. 2020-03-07 00:32:04 +01:00
ThePendulum 6820608a09 Gamma won't attempt to fetch photos when 'photos' parameter is false. 2020-03-06 23:50:52 +01:00
ThePendulum aad91fe4ae Added Girls Under Arrest to Adult Time. Added mobile album support to some Girlsway sites. Fixed Moms On Moms link. 2020-03-06 23:40:15 +01:00
ThePendulum a579e3c88f Enabled mobile album scraping for Burning Angel and Fame Digital. 2020-03-06 23:08:55 +01:00
ThePendulum 5d468b58fd Enabled mobile album fetching for Pure Taboo and Fantasy Massage. 2020-03-06 19:44:20 +01:00
ThePendulum 96eef822d6 Allowing mobile parameter to be set on Gamma network. Added mobile to 21Naturals and 21Sextreme, moved 21Sextury mobile to network. 2020-03-06 19:26:23 +01:00
ThePendulum 5c63bd860b Tags module filters out undefined tags. Gamma will return [] instead of undefined when no tags are available. 2020-03-06 18:59:32 +01:00
ThePendulum 90172ea19a Added mobile album support to Gamma scraper. 2020-03-06 04:28:01 +01:00
ThePendulum db01599569 Added profile scraper with releases to Hush. Added qtexts to q to return text nodes individually. Including network in profile site. 2020-03-06 02:49:55 +01:00
ThePendulum 3889faee26 Added optional sequential scraping and acc release injection. Added Hush Pass and Interracial Pass logos. 2020-03-05 23:01:03 +01:00
ThePendulum fd6e90e74c Added tour layout scraper to Hush, enabling Interracial POVs, POV Pornstars and See Him Fuck. 2020-03-05 20:31:11 +01:00
ThePendulum 1407700511 Added Interracial Pass sites. Fixed Hush removing poster from base release. 2020-03-05 17:07:07 +01:00
ThePendulum 981b121980 Split Hussie Pass into Hush networks Hush Pass, Hussie Pass and Interracial Pass. 2020-03-05 05:07:14 +01:00
ThePendulum f10e4af29b Allowing scrapers to force channel allocation attempt. Added Hush Pass subsite handling to Hussie Pass scraper. 2020-03-05 03:44:27 +01:00
ThePendulum 956afa6ae7 Added Hussie Pass scraper. 2020-03-05 02:47:52 +01:00
ThePendulum 15af3e91e0 Coalescing shoot ID in search. Added stop words for common TLDs. Sorting tags in search results. 2020-03-02 04:15:47 +01:00
ThePendulum e57f440665 Added Amateur Allure. 2020-03-01 05:28:08 +01:00
ThePendulum 3290a5f686 Focusing and auto-hiding search tooltip. Separated Girl Girl from Jules Jordan. 2020-03-01 00:21:54 +01:00
ThePendulum 8dd5925af6 Improved search engine query and added stop words. Added 'secondary' property to tag aliases, for tag aliases to be included in searches and alias lists. 2020-02-29 22:47:48 +01:00
ThePendulum f1f33080f6 Ignoring undefined video entropy. 2020-02-28 03:56:58 +01:00
ThePendulum 3dc8547431 Added fake data and Markov experiments. 2020-02-27 05:44:24 +01:00
ThePendulum 97cf5b2b6b Added seed file for test data. 2020-02-27 00:38:11 +01:00
ThePendulum 800a25743d Storing image dimensions and file size to database. Added new site Filthy Femdom to Kink. 2020-02-24 04:01:58 +01:00
ThePendulum 6d1f30f703 Passing matching site to profile scrapers. Allowing scrapers to pass avatar metadata. Added scraper and copyright properties to media. Auto-adding copyright from site or scraper to avatars. Separated Porn Pros from Whale Member. 2020-02-24 03:12:58 +01:00
ThePendulum 5acaa9c6bf Marked Metro HD sites as 'native' as to use site URL. 2020-02-23 22:07:35 +01:00
ThePendulum 0ede7e0f82 Added various Porn Pros archive sites. 2020-02-23 05:00:08 +01:00
ThePendulum e5c6ccd252 Scraping upcoming Vixen scenes. Fetching release media groups sequentially to prevent collisions. 2020-02-22 04:37:48 +01:00
ThePendulum 349a5a506e Queueing and batching media HTTP requests for improved reliability. 2020-02-22 03:22:30 +01:00
ThePendulum 13a8221b92 Added New Sensations. Returning null from q's date formatter when date is invalid. 2020-02-21 03:51:57 +01:00
ThePendulum 377970f874 Added parent-child relations to network, showing parent network in sidebar. Added Burning Angel using Gamma API. 2020-02-20 02:35:23 +01:00
ThePendulum 97f5e49187 Refactored media module. Returning 320p and 720p videos from MindGeek as teasers instead of trailers. 2020-02-19 04:41:53 +01:00
ThePendulum 40bf476ea6 Fixed Porn Pros scraper. Added various Score site logos. 2020-02-18 16:00:36 +01:00
ThePendulum 880cc64022 Added Full Porn Network scraper. 2020-02-15 01:50:22 +01:00
ThePendulum fe3cc901b0 Added PG site search function and enabled site search bar. Added Full Porn network assets. 2020-02-15 01:10:32 +01:00
ThePendulum 5e461f1f1a Added proper Gamma parameters to 21Sextury sites. 2020-02-13 23:05:28 +01:00
ThePendulum 1ff8d37d89 Added upcoming support to Nubiles. Renamed q's formatDate to extractDate, added actual formatDate. 2020-02-13 04:11:32 +01:00
ThePendulum c9c7a33585 Added Cherry Pimps/Pimp.XXX. 2020-02-13 03:44:04 +01:00
ThePendulum b7abd805e2 Added Girl Girl to Jules Jordan. 2020-02-12 23:49:22 +01:00
ThePendulum cd2ca65903 Added Nubiles network. 2020-02-12 23:00:32 +01:00
ThePendulum 98d29c0af0 Added Insex fallback for Paintoy. 2020-02-12 16:26:08 +01:00
ThePendulum 48b10d0f49 Using brief mode on Insex. 2020-02-12 04:59:15 +01:00
ThePendulum b8074205ef Added Insex. Renamed q's stand-alone date function. Separated q's trim function. Release tile uses cover if available, and poster is not available. 2020-02-12 04:39:57 +01:00
ThePendulum fa30366ea4 Moved site tags from tag seed to site seed. 2020-02-12 02:19:43 +01:00
ThePendulum 2f70de8e11 Added preflight method to scrapers. Added Assylum. 2020-02-12 01:54:54 +01:00
ThePendulum df4a2b0bb3 Fetching old scene links from Vivid search API. 2020-02-11 19:05:12 +01:00
ThePendulum dd6a1d9bfd Added Vivid network. Added ASMR Fantasy to Adult Time. Storing deep URL in database. Added href to header links. 2020-02-11 04:58:18 +01:00
ThePendulum ce448da7e0 Added 21 Naturals and 21 Sextreme sites. 2020-02-10 23:11:11 +01:00
ThePendulum 139f0ce7cb Allowing release scrapers to return actor details. Added True Amateurs. 2020-02-09 23:25:54 +01:00
ThePendulum 0f513266a0 Added Black for Wife to JayRock. Switched parameters field to JSON type. 2020-02-09 19:41:39 +01:00
ThePendulum e61ed2bb5f Added 21Sextreme and 21Naturals networks. Scraping all actor release pages for Brazzers. 2020-02-09 02:01:39 +01:00
ThePendulum 8ca458a8ff Added Moms on Moms to Girlsway. 2020-02-08 05:16:04 +01:00
ThePendulum d2cb74a252 Added Fantasy Massage sites. Improved Private scraper, added movie link. 2020-02-08 04:52:32 +01:00
ThePendulum 1546e0836c Split Girlsway from Adult Time. Added Fantasy Massage. Using Gamma scraper for Pure Taboo. Added photo path parameter to Gamma scraper. 2020-02-08 02:49:39 +01:00
ThePendulum 5ba308f07a Added Adult Time. Adding context to logger. 2020-02-07 19:53:16 +01:00
ThePendulum 6b56bad21e Added Twistys. 2020-02-07 04:44:01 +01:00
ThePendulum 30963b94dd Added profile releases for classic Fame Digital sites (Silvia Saint and Silverstone DVD). 2020-02-07 03:40:11 +01:00
ThePendulum 631ac34573 Blowpass now uses Gamma module for latest and upcoming. 2020-02-07 01:06:39 +01:00
ThePendulum 65f98c6387 Refactored 21Sextury module to use Gamma API. 2020-02-06 23:51:13 +01:00
ThePendulum 6e1de52a40 Added Fame Digital. Added actor release scraping to DDF Network. Improved q and Gamma scraper. 2020-02-06 23:15:28 +01:00
ThePendulum 38bab672d4 Better handling of empty profiles. Modified Score scraper for Big Tit Terry Nova. Improved Naughty America and various other logos. 2020-02-05 01:41:30 +01:00
ThePendulum f921bb4ae9 Generating and using URL slugs for releases, improver slugify module. Added 'extract' parameter to MindGeek scraper to get scenes not associate with a channel (see Digital Playground). Added various high res logos. 2020-02-04 03:12:09 +01:00
ThePendulum ef602a3a15 Added basic filename copy. Added HTTP helper to q. Fetching all actor release pages from Naughty America. Added various high res network logos. 2020-02-04 00:18:53 +01:00
ThePendulum 5bae5b6e5f Adapted Score scraper for 18eighteen, 40 Something Mag and 50 Plus Milfs. Updated Score network logo and favicon. 2020-02-03 02:57:53 +01:00
ThePendulum a671190fff Adapted Score scraper for Score Classics. 2020-02-03 02:04:47 +01:00
ThePendulum a45bebddac Adapter Score scraper for Score Videos. 2020-02-03 00:39:43 +01:00
ThePendulum 0ed1b2eff9 Added generic photo page extract method to media module, to allow pre-filtering sources and relief Dogfart scraper. Added 'transsexual' site tag to Trans Angels. 2020-02-02 22:36:33 +01:00
ThePendulum a97c6defca Added teaser support. Added Score network with scraper for Scoreland. Improved q. Added assets. 2020-02-02 05:14:58 +01:00
ThePendulum 94bf207397 Added Wicked network. Merged Evil Angel, XEmpire and Wicked into generic Gamma scraper. 2020-02-01 01:15:40 +01:00
ThePendulum b7f51a8deb Added avatar and actor releases to Bang Bros scraper. 2020-01-31 21:43:16 +01:00
ThePendulum 504bcd02e3 Added Naughty America profile and actor releases scraper. 2020-01-31 19:25:42 +01:00
ThePendulum 3f113310e3 Added Trans Angels to MindGeek. Interpreting MindGeek 'other' gender as transsexual. 2020-01-31 00:25:51 +01:00
ThePendulum ff61094b69 Added Men network and Icon Male to MindGeek. Added entropy filter to media module to help filter out generic avatars. Added Pure Taboo. Various logo updates. 2020-01-30 01:14:31 +01:00
ThePendulum d61ea26eb5 Added 'ignore' parameter to sites. Added light logo for Metro HD. 2020-01-29 04:02:32 +01:00
ThePendulum 8b840f52d1 Added description property to LegalPorno scraper (rarely used, see GIO337). 2020-01-29 03:19:38 +01:00
ThePendulum fc675ae144 Added Metro HD network using MindGeek scraper. Fixed MindGeek profile scraper avatar issue. 2020-01-29 02:24:19 +01:00
ThePendulum 76852daf6d Added VogoV (no trailer yet). Fixed MindGeek profile scraper. 2020-01-28 03:05:53 +01:00
ThePendulum 9db0e97512 Improved actor profile flow. Added images. 2020-01-23 03:52:12 +01:00
ThePendulum f8175f6054 Added generic Gamma photo and actor scraper for XEmpire, 21Sextury, Blowpass and Evil Angel. 2020-01-22 22:25:58 +01:00
ThePendulum 4074a5bb64 Added Mile High Media scraper. 2020-01-16 21:56:33 +01:00
ThePendulum 61fee5e4f6 Added q scraping helper. Added Perfect Gonzo scraper. 2020-01-16 21:04:44 +01:00
ThePendulum 5dda81535d Added PornPros satellite sites. 2020-01-15 01:01:16 +01:00
ThePendulum 75c53d338a Added Porn Pros sites and scraper. 2020-01-14 21:45:30 +01:00
ThePendulum 9255467091 Refactored MOFOS scraper to use generic MindGeek scraper. Added Digital Playground and Fake Hub. 2020-01-14 19:56:46 +01:00
ThePendulum ef76909d3c Merged Reality Kings and Babes into MindGeek scraper. Kept classic latest wrapper for Look At Her Now and Tranny Surprise. 2020-01-14 04:50:42 +01:00
ThePendulum 8c76568f44 Added Babes network 2020-01-14 01:50:11 +01:00
ThePendulum 859cb7e1f3 Added support for Family Strokes. 2020-01-13 23:45:09 +01:00
ThePendulum 97cb4a061e Added Team Skeet scraper. 2020-01-12 05:30:46 +01:00
ThePendulum 8f72e61928 Added Jay Rock network. Fixed site logo not showing up without URL. 2020-01-11 05:31:04 +01:00
ThePendulum f572f38b5c Added Jay's POV scraper. 2020-01-10 22:10:11 +01:00
ThePendulum 863df38b29 Simplified tag media seed file. Renamed tags. Media refactor experiment. 2020-01-08 23:33:24 +01:00
ThePendulum a773e0bf54 Added Manuel Ferrara site to Jules Jordan. Refactored Jules Jordan photo scraper for better compatability and quality. 2020-01-08 05:12:14 +01:00
ThePendulum 0a19f2e624 Added Bang! deep scrape. Improved network page layout. Added Bang Bros logos. 2020-01-07 04:23:28 +01:00
ThePendulum 9e6e5605ed Added favicons for all networks. Showing sites on dark background, lightened logos. Updated Kink logos. Changed Bang! studios to sites, removed studio prominence. 2020-01-06 21:49:37 +01:00
ThePendulum 5b118446ae Added Kosovo to countries. 2020-01-06 14:55:55 +01:00
ThePendulum bb5b8085f6 Added Bang! scraper. Showing studio on tile. Added favicons to tiles. 2020-01-06 05:19:38 +01:00
ThePendulum 30cf597ec9 Improving network overview. Added DDF logos and tag posters. 2020-01-04 04:58:56 +01:00
ThePendulum 72b175e9e2 Removed views in favor of PostGraphile filter and sort plugins. Updated site modules to GraphQL. Added tag posters. 2020-01-04 02:51:58 +01:00
ThePendulum 1c43884102 Refactoring client to reflect database changes. 2019-12-19 04:42:50 +01:00
ThePendulum 31aee71edb Normalized database. Updated seed files. Simplified seed upsert. 2019-12-19 02:35:07 +01:00
ThePendulum 6950a76cb5 Abstracted fragments and curation. Using GraphQL for tags. 2019-12-16 05:30:25 +01:00
ThePendulum 7ba716cd6f Experimenting using GraphQL in favor of REST. 2019-12-15 05:42:51 +01:00
ThePendulum a03a00f5d9 Added Sperm Swallowers and The Ass Factory to Jules Jordan scraper. Added photo source. 2019-12-12 05:18:43 +01:00
ThePendulum a310f9bb1d Added support for release photo fallbacks. Limited photo fetching concurrency. Modifier XEmpire scraper for AllBlackX support and improved photo scraping. Added movie property to Evil Angel scraper. 2019-12-12 03:12:05 +01:00
ThePendulum f251b0b17c Updated DAP and DATP posters. 2019-12-11 05:20:38 +01:00
ThePendulum b9af7255f6 Fixed release actor origin bug. 2019-12-11 05:15:23 +01:00
ThePendulum 18f37e3fbd Moved all tag photos to public asset directory. 2019-12-11 02:25:25 +01:00
ThePendulum 85e55eebaf Added Kelly Madison Media (Teen Fidelity) release scraper. 2019-12-10 05:10:08 +01:00
ThePendulum 04a89efa58 Refactored 21sextury scraper. 2019-12-09 05:00:49 +01:00
ThePendulum 222260d3cf Generalized tags to allow them to be assigned to anything. 2019-12-07 04:41:16 +01:00
ThePendulum 98b9c885ec Added support for Reality Kings sites 'Look At Her Now' and 'Tranny Surprise'. 2019-12-07 00:42:47 +01:00
ThePendulum a2925febeb Moved tag posters back to img/, as they are part of the project. Still using database to track which tags have a poster. 2019-12-06 05:09:08 +01:00
ThePendulum 71cb85c3e1 Allowing scrapers to return raw tags and site URLs or slugs, to gradually remove site and tag fetching from individual scrapers. Added media and deep fetchin support to Perv City scraper. 2019-12-05 01:26:22 +01:00
ThePendulum 55e3130062 Move tag posters and photos to media database. 2019-12-04 21:58:08 +01:00
ThePendulum bead69de49 Added basic actor and network overview. Added DDF Network actor scraper. Various bug fixes and layout improvements. 2019-11-30 05:55:32 +01:00
ThePendulum 0dbe853f39 Resolving actor birth and residence place before storage. Layout improvements. 2019-11-29 05:46:06 +01:00
ThePendulum d113123778 Fixed countries seed file. Updated MOFOS scraper. Improved Reality Kings scraper. Limiting photos for XEmpire scraper. 2019-11-27 04:58:38 +01:00
ThePendulum e8130c3634 Scraping actor profiles from FreeOnes. 2019-11-17 03:56:45 +01:00
ThePendulum 2857387441 Added bash CLI shorthand. Properly setting parameters in site results. Removed obsolete URL restriction from Reality Kings scraper. 2019-11-16 23:37:33 +01:00
ThePendulum 1c3f17ec82 Added timerange filters. Refactored releases module for more efficient queries. 2019-11-15 01:27:58 +01:00
ThePendulum bf223adc55 Redesigned release page. Added 'single site' marker, linking directly to network page. Fixed Dogfart scraper duration. 2019-11-14 05:13:38 +01:00
ThePendulum 117d669a69 Added Private Black, improved Private scraper to handle their non-private.com sites. 2019-11-14 00:00:29 +01:00
ThePendulum f9f9cc7977 Added tag filter to releases query, enabled on homepage. 2019-11-13 03:14:24 +01:00
ThePendulum 832e96ced1 Improved site and network pages. Fixed various issues. 2019-11-12 01:22:20 +01:00
ThePendulum 681f1bca85 Added tag page. Added default 'anal' tag to Vixen scraper for Tushy and Tushy Raw. 2019-11-11 05:18:28 +01:00
ThePendulum e9277028e2 Added Mike Adriano scraper (True Anal, All Anal, Swallowed, Nympho). Updated README. 2019-11-09 01:22:50 +01:00
ThePendulum 0567b1b2ff Refactored seed files to use programmatic upsert instead of SQL duplicate ignore. 2019-11-06 02:47:10 +01:00
ThePendulum c466d791f5 Removed upsert from store releases, observing effects; fixes ? inserted as . Removed query from stored Dogfart URLs. 2019-11-05 00:54:47 +01:00
ThePendulum 5745cd33d8 Added Dogfart scraper. Added 'date added' property to release page. 2019-11-04 05:47:37 +01:00
ThePendulum 2198c7ceb0 Added media support to Bang Bros scraper. Added untracked files. 2019-10-31 01:53:26 +01:00
ThePendulum 382e40b651 Added studio support. Added studios for LegalPorno. Improved media fetch error handling. Fixed DDFNetwork scraper, added media support. 2019-10-30 04:45:42 +01:00
ThePendulum f7f5e8a306 Deriving transsexual tag from actors in Evil Angel scraper. 2019-10-28 02:12:11 +01:00
ThePendulum 0e784a274f Reworked tag matching. Added tag filter groundwork. 2019-10-28 00:58:54 +01:00
ThePendulum 2c32ed6549 Completed Evil Angel scraper. Added trans tags. 2019-09-28 03:26:04 +02:00
ThePendulum b12d74cbb3 Fixed Vixen scraper, site restructured. Added Deeper to Vixen network. 2019-09-25 02:44:25 +02:00
ThePendulum cf4987ba48 Using media database for images. 2019-09-10 16:48:04 +02:00
ThePendulum 926c227b85 Refactored database to separate IDs from slugs 2019-09-08 03:53:09 +02:00
ThePendulum e3558fc0c5 Using grid layout with thumbnails. 2019-05-08 05:50:13 +02:00
ThePendulum 3df5a821b6 Saving results to database. Showing webpage. 2019-05-06 02:01:57 +02:00
ThePendulum b486b6abdd Added Bang Bros scraper. 2019-04-12 03:12:53 +02:00
ThePendulum bc01cbe6dc Added Bang Bros data (no scraper yet). Added 21Sextury scene fetch date fallback. 2019-04-10 03:42:20 +02:00
ThePendulum be74f84781 Expanded README. 2019-04-08 04:07:39 +02:00
ThePendulum 763ffb341a Added Naughty America scraper. 2019-04-08 03:07:16 +02:00
ThePendulum d526e4daea Added Reality Kings scraper. Improved site finder. 2019-04-08 01:49:45 +02:00
ThePendulum ec056a177a Added 21Sextury scraper. Various improvements. 2019-04-07 20:51:14 +02:00
ThePendulum c8abb302e4 Added DDFNetwork scraper. Modified tag matching query to be case insensitive. 2019-04-07 05:01:06 +02:00
ThePendulum 885a815022 Added tag groups. Added MOFOS scraped. Improved entry ID handling. 2019-04-07 02:15:57 +02:00
ThePendulum 3a90f98d41 Added Blowpass scraper. Split shootId and pageId. 2019-04-06 23:24:26 +02:00
ThePendulum 2b818e379a Scrapers can now iterate through pages. Filtering unique releases before saving to database. Improved scrapers and rendering. 2019-04-05 03:45:40 +02:00
ThePendulum 6190247ac5 Added Brazzers scraper. Removed scene-only sites in database in favor of automatic network fallback. 2019-04-04 20:22:47 +02:00
ThePendulum 784542253b Expanded tag database. 2019-04-04 04:44:37 +02:00
ThePendulum b3beeef3e4 Added Private scraper. Added Vixen scraper to repository. 2019-04-04 04:00:28 +02:00
ThePendulum 439d3225ec Added Vixen scraper. Added LegalPorno studio IDs. 2019-04-01 02:45:15 +02:00
ThePendulum 8421cd8648 Added parameters column to sites database, fixes Perv City scraper. Getting shoot ID from all existing scrapers. 2019-03-26 01:26:47 +01:00
ThePendulum 4d7f323e86 Replaced network and tag files with SQLite database. 2019-03-25 03:57:33 +01:00