Ignore profile source configuration 'try until success' when scraping actor scenes #21

New Issue

pendulum · 2020-11-26T02:54:47Z

pendulum commented

2020-11-26 02:54:47 +00:00

Profile sources can be defined as a nested array to indicate that instead of fetching a profile from each source, it should only fetch a profile from the first source that returns one. This allows sites with the same bio database to be grouped together, and prevents duplicate avatars and skewed profile interpolation (if 10 sites use the same database, one true source claiming blue eyes versus 3 true sources claiming brown eyes, could become 10 duplicate sources claiming blue eyes).

However, this approach does not allow all scenes to be scraped from an actor when --actor-scenes is used.

Find another way to discard the duplicate biographies and avatars from grouped profile sources, so the scenes can be scraped.

Profile sources can be defined as a nested array to indicate that instead of fetching a profile from each source, it should only fetch a profile from the first source that returns one. This allows sites with the same bio database to be grouped together, and prevents duplicate avatars and skewed profile interpolation (if 10 sites use the same database, one true source claiming blue eyes versus 3 true sources claiming brown eyes, could become 10 duplicate sources claiming blue eyes). However, this approach does not allow all scenes to be scraped from an actor when `--actor-scenes` is used. Find another way to discard the duplicate biographies and avatars from grouped profile sources, so the scenes can be scraped.

pendulum added the

enhancement

label 2020-11-26 02:54:47 +00:00

pendulum self-assigned this 2020-11-26 02:54:47 +00:00

Sign in to join this conversation.