Merge pull request #12 from norbertjoni/main

Fix the whitelist log and add ping health check

README.md

@@ -18,6 +18,8 @@ Cloudflare Gateway allows you to create custom rules to filter HTTP, DNS, and ne
 - Automatically cleans up filter lists: removes duplicates, invalid domains, comments and more
 - Works fully unattended
 - Whitelist support, allowing you to prevent false positives and breakage by forcing trusted domains to always be unblocked.
+- Optional health check: Sends a ping request ensuring continuous monitoring and alerting for the workflow execution.
+
 
 ## Usage
 
@@ -53,6 +55,8 @@ Please note that the GitHub Action downloads the recommended blocklists and whit
 - `CLOUDFLARE_ACCOUNT_ID`: Your Cloudflare account ID
 - `CLOUDFLARE_ACCOUNT_EMAIL`: Your Cloudflare account email
 - `CLOUDFLARE_LIST_ITEM_LIMIT`: The maximum number of blocked domains allowed for your Cloudflare Zero Trust plan. Use 300000 for the free plan or if you're unsure.
+- `PING_URL`: /Optional/ The HTTP(S) URL to ping (using curl) after the GitHub Action has successfully updated your filters. Useful for monitoring.
+
 
 3. Create a new file in the repository named `.github/workflows/main.yml` with the contents of `auto_update_github_action.yml` found in this repository. The default settings will update your filters every week at 3 AM UTC. You can change this by editing the `schedule` property.
 4. Enable GitHub Actions in your repository settings.

auto_update_github_action.yml

@@ -56,3 +56,11 @@ jobs:
         CLOUDFLARE_ACCOUNT_EMAIL: ${{ secrets.CLOUDFLARE_ACCOUNT_EMAIL }}
         CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
         CLOUDFLARE_LIST_ITEM_LIMIT: ${{ secrets.CLOUDFLARE_LIST_ITEM_LIMIT }}
+
+    - name: Send ping request
+      if: env.PING_URL != ''
+      working-directory: cloudflare-gateway-pihole-scripts
+      env:
+        PING_URL: ${{ secrets.PING_URL }}
+      run: |
+        curl "${{ env.PING_URL }}"

cf_list_create.js

@@ -38,10 +38,10 @@ fs.readFile('whitelist.csv', 'utf8', async (err, data) => {
     }).filter(domain => {
       return domainValidationPattern.test(domain);
     });
+    console.log(`Found ${whitelist.length} valid domains in whitelist.`);
   }  
 });
 
-console.log(`Found ${whitelist.length} valid domains in whitelist.`);
 
 // Read input.csv and parse domains
 fs.readFile('input.csv', 'utf8', async (err, data) => {
@@ -90,7 +90,11 @@ fs.readFile('input.csv', 'utf8', async (err, data) => {
 
   // Remove domains from the domains array that are present in the whitelist array
   domains = domains.filter(domain => {
-    return !whitelist.includes(domain);
+    if (whitelist.includes(domain)) {
+      console.warn(`Domain found in the whitelist: ${domain} - removing`);
+      return false;
+    }
+    return true;
   });
 
   // Trim array to 300,000 domains if it's longer than that