Rationale:

  • The db currently tracks “has_captcha”. Cloudflare influences that regardless of whether the site is itself configured to push a CAPTCHA.
  • The db currently tracks “open_registrations”. The degree of openness depends on whether a node is on Cloudflare, regardless of how the software is configured.
  • Checking for Cloudflare is free. That is, no extra connections are needed. It’s simply a matter of checking for the presence of CF-RAY in the http headers on existing fetches.
  • The mission purpose of fediseer is to facilitate trust assessment. The use of Cloudflare is a significant factor in that regard.
        • LibreMonk@linkage.ds8.zoneOPEnglish
          1·
          7 days ago

          It’s still down for me. Just times out. So I have no idea what you propose.

          Anyway, I don’t imagine how a workaround would be possible. The dataset is missing the data. If I have to scrape for the data myself, it defeats the point of a public dataset. I could write in my code a Cloudflare check, but that doesn’t help anyone else using the fediseer dataset.

            • LibreMonk@linkage.ds8.zoneOPEnglish
              1·
              1 day ago

              Thanks! Codeberg.org gives my browser problems, but I was able to shallow clone that over Tor using git.

              My first hack was to add a --socks4a parameter to all the curl calls to force it over Tor.

              It did not work out of the box. I could see that it sources config.sh. The example file shows API keys. The instances_list.json file shows {"error":{"message":"Invalid Authorization header."}}, which implies that I need an API key. If I simply run this:

              curl --socks4a 127.0.0.1:9050 -o "${filename}" 'https://fediseer.com/api/v1/all_instances'

              there is no problem. I suppose you have hard-coded an API key to do operations that submit data, which is then used for getting data as well. Anyway, the script works for checking a single instance.

              I wrote a script that uses Cloudflare’s ASNs to grab all its IP ranges. Then the script checks without even connecting to the target just by resolving the IP and checking if it matches CF’s range. It’s still just a messy workaround. It’s really something that should be in the fediseer dataset.