Skip to main content
HashnodeTodd H. Gardner

Command Palette

Search for a command to run...

From Console Error to Root Cause: A Debugging War Story

Published
•7 min read
T
Todd H. Gardner

Friday, 4:47 PM. Support forwards a screenshot: "SyntaxError: Unexpected token <". Your weekend just got interesting.

I've been building and debugging web applications for over 15 years, and I still remember this particular bug. Not because it was technically complex, but because it perfectly captures how modern web debugging really works: it's detective work where the obvious answer is usually wrong.

Let me walk you through exactly how this unfolded.

The Crime Scene

The screenshot from the customer showed a white screen and a console error:

SyntaxError: Unexpected token '<'
    at JSON.parse (<anonymous>)
    at response.json (main.js:482:18)
    at processResponse (main.js:447:22)

My first thought: "Classic. The API is returning HTML instead of JSON."

I'd seen this Unexpected token error pattern hundreds of times. Usually it means a 503 page or login redirect. Should be a quick fix. I'd be home by 5:15.

I was wrong.

The First Dead End

I opened our monitoring dashboard. No API errors. No 503s. The backend was healthy. Load balancers were fine. Response times were normal.

Interesting.

I pulled up the customer's session in our error tracking. Here's what I saw in the network telemetry:

16:44:31 GET /api/user/profile 200 OK (142ms)
16:44:31 GET /api/dashboard/stats 200 OK (89ms)
16:44:32 GET /api/notifications 200 OK (234ms)
16:44:32 ERROR SyntaxError: Unexpected token '<'

All the API calls succeeded. They all returned 200 OK. So where was the HTML coming from?

Going Deeper

I needed to reproduce this locally. I had the customer's user ID, so I impersonated their account in our staging environment. Everything worked fine.

But here's the thing about debugging production issues: staging is a liar. It doesn't have the same network topology, the same load balancers, the same CDN configuration. It's a pale imitation of production.

So I did something you're not supposed to do: I added temporary debug logging to production.

// Temporary debug logging - DO NOT LEAVE IN PRODUCTION
async function processResponse(response) {
  const contentType = response.headers.get('content-type');
  const responseText = await response.text();

  // Log everything for user 842917
  if (window.userId === '842917') {
    console.log('Response details:', {
      url: response.url,
      status: response.status,
      contentType: contentType,
      bodyPreview: responseText.substring(0, 200)
    });
  }

  // Try to parse as before
  try {
    return JSON.parse(responseText);
  } catch (e) {
    console.error('Parse failed for:', responseText);
    throw e;
  }
}

Then I waited. The customer was on the west coast, so they'd likely hit the issue again around 2 PM Pacific when they got back from lunch.

The Plot Twist

At 2:23 PM, boom. New error logged. But the debug output was not what I expected:

Response details: {
  url: "https://api.myapp.com/notifications",
  status: 200,
  contentType: "application/json",
  bodyPreview: '<!DOCTYPE html><html><head><title>Welcome to AT&T Free WiFi</title>...'
}

Wait. What?

The response URL was our API. The status was 200. The Content-Type header said JSON. But the body was... an AT&T WiFi login page?

The Real Culprit

The customer was working from a coffee shop. The AT&T WiFi portal was doing something I'd never seen before: intercepting HTTPS API calls, returning 200 OK with the original Content-Type headers, but replacing the body with their login page.

This is insane behavior for a captive portal, but here we were.

Most captive portals just block requests or redirect them. This one was performing a man-in-the-middle attack, keeping the headers but swapping the body. It was probably some "smart" optimization to avoid breaking single-page apps, but it had the opposite effect.

The Investigation Continues

Now I was curious. How widespread was this problem? I dug into our error logs with a new query:

SELECT 
  COUNT(*) as occurrences,
  EXTRACT(hour FROM timestamp) as hour_of_day,
  user_agent,
  ip_subnet
FROM error_logs
WHERE 
  message LIKE '%Unexpected token%'
  AND message LIKE '%<%'
  AND timestamp > NOW() - INTERVAL '30 days'
GROUP BY hour_of_day, user_agent, ip_subnet
ORDER BY occurrences DESC

The pattern was clear:

  • 73% of these errors happened between 7-9 AM and 12-2 PM (coffee shop hours)

  • They clustered around specific IP ranges (public WiFi providers)

  • They disproportionately affected our mobile web users

We were losing customers to coffee shop WiFi.

Building the Fix

The naive fix would be to detect the HTML and show an error. But that's not helpful to someone trying to work from a Starbucks. They can't fix the WiFi.

Here's what we actually built:

class NetworkInterceptorDetector {
  constructor() {
    this.interceptorDetected = false;
    this.checkEndpoint = '/api/health';
  }

  async checkForInterception() {
    try {
      const response = await fetch(this.checkEndpoint, {
        cache: 'no-store',
        headers: {
          'X-Intercept-Check': Date.now().toString()
        }
      });

      const text = await response.text();

      // Check if response looks like HTML
      if (text.trim().startsWith('<')) {
        this.handleInterception(text);
        return true;
      }

      // Check if response is our expected health check
      try {
        const json = JSON.parse(text);
        if (!json.healthy) {
          throw new Error('Invalid health response');
        }
      } catch (e) {
        this.handleInterception(text);
        return true;
      }

      return false;
    } catch (error) {
      // Network errors are different from interception
      console.error('Network check failed:', error);
      return false;
    }
  }

  handleInterception(responseText) {
    this.interceptorDetected = true;

    // Check if it's a known portal
    const portalType = this.identifyPortal(responseText);

    // Show user-friendly message
    this.showPortalWarning(portalType);

    // Try to open portal page
    if (portalType.loginUrl) {
      this.offerPortalRedirect(portalType.loginUrl);
    }
  }

  identifyPortal(html) {
    const patterns = [
      { regex: /AT&T.*WiFi/i, name: 'AT&T WiFi', loginUrl: 'http://neverssl.com' },
      { regex: /Xfinity.*WiFi/i, name: 'Xfinity WiFi', loginUrl: 'http://neverssl.com' },
      { regex: /Starbucks/i, name: 'Starbucks WiFi', loginUrl: 'http://neverssl.com' },
      { regex: /Airport.*WiFi/i, name: 'Airport WiFi', loginUrl: null }
    ];

    for (const pattern of patterns) {
      if (pattern.regex.test(html)) {
        return pattern;
      }
    }

    return { name: 'Public WiFi', loginUrl: 'http://neverssl.com' };
  }

  showPortalWarning(portalType) {
    // Create non-blocking notification
    const banner = document.createElement('div');
    banner.className = 'network-warning-banner';
    banner.innerHTML = `
      <div class="warning-content">
        <strong>⚠️ ${portalType.name} Login Required</strong>
        <p>The WiFi network requires authentication. Our app may not work correctly until you sign in.</p>
        ${portalType.loginUrl ? 
          `<button onclick="window.open('${portalType.loginUrl}', '_blank')">
            Open WiFi Login Page
          </button>` : ''
        }
        <button onclick="this.parentElement.parentElement.remove()">Dismiss</button>
      </div>
    `;
    document.body.prepend(banner);
  }

  offerPortalRedirect(url) {
    // Some portals only appear on non-HTTPS URLs
    setTimeout(() => {
      if (this.interceptorDetected) {
        console.log('Tip: Visit http://neverssl.com to trigger WiFi login');
      }
    }, 3000);
  }
}

// Check on app startup and after network errors
const detector = new NetworkInterceptorDetector();

window.addEventListener('online', () => {
  detector.checkForInterception();
});

// Also check when API calls fail
async function apiCall(url, options) {
  try {
    const response = await fetch(url, options);
    const contentType = response.headers.get('content-type');

    if (contentType?.includes('json')) {
      return await response.json();
    } else {
      // Might be intercepted
      const text = await response.text();
      if (text.trim().startsWith('<')) {
        await detector.handleInterception(text);
        throw new Error('Network requires authentication');
      }
      // Try to parse anyway
      return JSON.parse(text);
    }
  } catch (error) {
    if (error.message.includes('Unexpected token')) {
      // Check for interception
      await detector.checkForInterception();
    }
    throw error;
  }
}

The Lessons Learned

This bug taught me several things:

1. The network is hostile and weird. We assume networks just pass our data through. They don't. They modify, intercept, inject, and mangle our traffic in creative ways.

2. Error messages are just the starting point. That "Unexpected token" error was technically correct but completely misleading about the root cause.

3. Production debugging requires production access. You can't debug real-world issues in the safety of staging. Sometimes you need to get your hands dirty with production logs.

4. Users can't tell you what's wrong. Our customer reported "the app doesn't work." They had no idea their coffee shop WiFi was the problem. We had to be detectives.

5. The fix isn't always fixing the bug. We couldn't fix AT&T's WiFi. We had to work around it and help users understand what was happening.

The Aftermath

After deploying the detection system:

  • JSON parsing errors dropped by 84%

  • Support tickets about "blank screens" dropped by 60%

  • We discovered 12 different WiFi portals doing similar interception

  • We added a "connection health" indicator to the app

But the best part? Three weeks later, I got an email from the original customer:

"Hey, I don't know what you did, but the app works great at coffee shops now. That little WiFi warning thing is super helpful. Thanks!"

They never knew about the hours of debugging, the production patches, or the detective work. They just knew it worked.

And that's what debugging is really about. Not the clever solution or the complex root cause analysis. It's about making things work for real users in the real world, even when that world is doing absolutely insane things to your perfectly reasonable code.

Your Debugging Toolkit

If you're facing similar mysterious production issues, here's your toolkit:

  1. Never trust the obvious answer - Dig deeper than the error message

  2. Add tactical logging - Sometimes you need temporary production debugging

  3. Look for patterns - Time of day, user location, and device type are clues

  4. Test in hostile conditions - Use public WiFi, slow connections, and old devices

  5. Build defensive code - Assume the network is trying to break you

  6. Help users help themselves - Show them what's wrong in terms they understand

The next time you see an "Unexpected token <" error, remember: it might not be your API returning HTML. It might be a coffee shop in San Francisco doing something weird with WiFi.

Happy debugging, and may your production errors be obvious and your root causes be simple.

P.S. - If you're tired of debugging production issues through customer screenshots and want real visibility into what's happening, you should check out TrackJS. We built it specifically because of war stories like this one.

#javascript#debugging#devops#apis

Comments

Join the discussion

No comments yet. Be the first to comment.