Prologue

I discovered Calyx while stumbling about in the DEF CON vendor village probably around 2018 - as someone rocking a phone plan with no data at the time, somehow I thought it made more sense to get an unlimited data hotspot instead of a slightly more expensive phone plan with enough data to tide me over when I wasn't on WiFi; I would remember it when I was waiting for a download to finish with my "25 Mbps" AT&T service. This year I finally had enough of AT&T gaslighting me about running some fiber by me and decided to give Calyx a shot - in the worst case I can sell my hotspot to someone else and get AT&T again, or (ugh) Comcast.

Onboarding Experience

Buying a hotspot from Calyx was pretty effortless; I meant to pay in cash for a fully anonymous hostpot but I used a card like a normie instead. When I got back home I installed the special Calyx SIM, charged the hotspot's battery and turned it on...only to find out T-Mobile had decided to finalize its migration of Sprint customers onto their network, effectively preventing any of whom from activating new devices or connecting to the network - and since Calyx is essentially a Mobile Citizen reseller, who got their start thanks to a Sprint deal for nonprofits, I was now stuck.

Thus began a full week of emailing Calyx, then emailing Mobile Citizen, then calling Mobile Citizen over multiple days until someone finally managed to get in touch with someone at T-Mobile to manually activate my hotspot. I think I checked the Calyx subreddit before I left for DEF CON just to confirm the service didn't start sucking immediately before I joined, [1] but the migration seemed to begin after I'd left. Calyx did release a statement acknowledging the issue after I got back, but it was one of those corporate statements that didn't give any kind of indication when things might improve and lacked any follow-up on what they're doing to fix the issue.

Overall 2/10 onboarding, I think my experience was exceptionally bad but I still don't like how it was handled and I think I would've been without service even longer had I not followed up so often.

SPEEEEED Testing

I ran some speed tests on my "25 Mbps" AT&T service before switching over and the performance matched my expectations:

When I drop the best/worst runs for each provider and average the rest, my download speed is only 11.7 Mbps with TestMy; otherwise AT&T is giving me 15/2 service with kinda high pings but no jitter.

As for Calyx, I made a top level contribution mainly for the hotspot with an ethernet port and support for running plugged in while not destroying its battery but it also meant I had access to 5G service which should provide a significant performance increase over AT&T. Given that the hotspot is about a mile from the closest tower as the crow flies, I expected decent performance; so how does it compare?

5G T-Mobile is pretty good. It roflstomps AT&T in terms of outright throughput, but the variance is surprisingly massive: TestMy has average speeds of 291.8/21.0 Mbps; MLab 350.1/22.7 Mbps; Ookla 361.9/23.0 Mbps with Clouvider, with down speeds close to 380 Mbps at other endpoints. Latency variance is also significantly increased over AT&T, with pings anywhere from 15 to 40 ms, but they seem to average in the same 23-26 ms range as AT&T. Jitter is also up significantly, but since I don't do much in terms of VoIP calls this is less of an issue to me.

The more eagle-eyed readers might have noticed an incongruity with the second set of speed test results - namely that Netflix upload speeds are almost an order of magnitude faster than the download speeds. This is due to T-Mobile throttling connections to video content providers; Netflix seems to top out at 2.5 Mbps.

Now these numbers are great for cosplaying a scientist and all, but they were all taken at roughly the same time and ISPs have been known to juice speedtest connections to look better than what they'll provide anywhere else on the internet; how real are these numbers? Thankfully, I run a speedtest tool on my router so I have some long-term connection statistics for AT&T:

This is to a speedtest provider but it's pretty clear the AT&T connection is solid - web surfing was generally a nonissue and I didn't have an issue with excessive YouTube buffering until I stepped up to 4K. However, as a computer-toucher I download more than my fair share of ISOs and they're not 700MB anymore - downloading at a sustained 15 Mbps makes me feel like I'm back in the dialup days without getting to enjoy hearing computers scream at each other. [2]

This is where I would pivot to talking about the Calyx service and show a long-term speedtest screenshot as well, but my router rebooted and baleeted its speed test data so you'll have to take my word the Calyx service isn't as consistent: download speeds have ranged from 40(!) to 340 Mbps, with generally high pings and jitter in the 4-8 ms range.

Large downloads are a lot better than they were on AT&T, but not as good as they should be: aside from the first couple days when I would see speeds as low as 40 Mbps, I have never noticed a download progress slower than 80 Mbps but I've also never seen a download progress much faster than 160 Mbps, a far sight from 300 Mbps. That 160 Mbps is also peak, not sustained - normally a download will settle around 100-120 Mbps before slowly creeping up to 160 Mbps, hold for a second or two and drop back down to 120 Mbps.

Usage Issues

It seems like every time I connect to a new site that uses Cloudflare I get hit with a CAPTCHA challenge - which is to say, it happens every time I visit a new site.

Remember when I said T-Mobile throttles connections to video content providers? I don't care much about Netflix since I pirate anything of theirs I find interesting, but YouTube is also throttled and as someone who watches far too many Youtube videos this has a significant QoL impact - videos would jump to 1080p on full screen basically immediately, but most of the videos I watch now struggle past 720p. The Calyx subreddit said switching APNs would bypass traffic shaping, but T-Mobile seems to have gotten wise; now users say only VPNs seem to work - speaking of...

My VPN experience has degraded to hot dogwater since the cutover - I assume the hotspot does something to outgoing UDP packets my VPN server doesn't like - but there's a "VPN Passthrough" option I haven't enabled yet so I need to get on with seeing what that does.

Additionally I have noticed videos will randomly take somewhere on the order of 10 seconds to go from initiating the connection to the first painting of the page, with no explanation as to why and no solution besides waiting it out.

As annoying as those issues are, the most annoying one is when the hotspot somehow enters a state where all attempted connections fail - I'll be surfing along without a care in the world when all of a sudden I lose internet access but instead of something like a timeout or connection error I get PR_END_OF_FILE_ERROR and I can't access anything until I restart the hotspot. The weird thing is I'll still be able to ping computers on the internet, so I knew the connection to the tower was still there - I haven't figured out the root cause yet, but my hunch is there's some sort of firewall or NAT table on the hotspot that reaches capacity.

On the subject of NAT, I might be double-natted? The hotspot supports passing through the IP address it receives from the tower to my router and I have ostensibly enabled the functionality, but the IP the router sees is 192.0.0.2 and the address I get from IP checking websites is completely different. For all I know, this might be working as expected; RFC 7335 says the range 192.0.0.0/29 should be used for "dummy" IPv4 addresses when performing IPv4/IPv6 translation.

While I'm on the subject of network chicanery, T-Mobile has done something with their network to make traceroute completely useless - my packets seem to fall off the face of the earth once they leave the hotspot unless I'm trying to traceroute to a DNS server.

I know I brought this up previously but you can feel the additional latency in a cellular connection in a way that doesn't really present itself during a speedtest, as shown by what happened to a local ping test when I first cut over to Calyx:

Checking in as I write this blog, things haven't improved much:

The hotspot overheated the first time I tried to use it; this could be because I stuck it in my attic in August which is hotter than Satan's taint, [3] but it might be even hotter next July. The AT&T modem never missed a beat, but since the outside of the hotspot is plastic there's no way to directly cool it if necessary; I rearranged the attic a bit to bring in an exhaust fan, but only time will tell if that'll be enough.

The hotspot has an option in its settings named "Allow factory reset from the touchscreen". As I'm setting up the hotspot the first time, I decide I only want to allow a factory reset with the button under the battery cover, so I disable the option. Lo and behold, I realize I have bungled some settings so I press the reset button and...nothing happens. Turns out, when you disable that option it also disallows factory resets from the hardware button! It would have taken less effort to write "Allow factory reset", which is what the option actually means, instead of "Allow factory reset from the touchscreen", which suggests non-touchscreen factory resets are still possible.

Conclusion

So, am I regretting my choice 2 months in? Not really - while there are a lot more hassles using Calyx and noticably decreased service quality, being over 10 times faster is enough to overlook many issues. Deducting points for the seemingly random "connection failures", bait-and-switched download speeds, YouTube throttling, significantly increased CAPTCHA challenges and (seemingly) garbage VPN support, I'd still give the service a 7/10 currently. I don't plan on selling my hotspot off just yet, but if AT&T actually laid some fiber near me I would definitely consider giving them another chance.