• 0 Posts
  • 15 Comments
Joined 1 year ago
cake
Cake day: March 20th, 2025

help-circle
  • We had an employee break procedure, make a dumb mistake, and cause ~$160k worth of damage to a mission-critical piece of infrastructure. It happened due to her own inattention and disregarding her “here’s how to shut down at the end of the night” checklist, at like 8PM. Basically, instead of doing steps A, B, C, and D, she went “eh I know what I’m doing,” jumped straight to step D, and suddenly heard very expensive noises. It required me and her supervisor to pull an overnight shift to get a bodged workaround in place, just to be ready for the next morning at 8AM. And even then, the gear was out of commission for about a month until we could get it fixed.

    All in all, it was about $80k worth of equipment repairs, $40k in equipment rentals (to keep things running in the meantime), and about $40k in additional labor (we had to hire specialized contractors to fix the gear).

    The employee 100% thought she was going to get fired when it happened. We were obviously angry and disappointed that she made such a dumb mistake, but we didn’t yell or chastise her. We simply told her to go ahead and clock out for the evening, and we’d deal with fixing things overnight. She tried to say she could stick around to help… But this was already at the end of her shift, she was obviously not in the right headspace to pull an overnight shift, and we were both too frustrated to have her around at the time. She was crying on her way out the door.

    The supervisor decided to keep her on instead of firing her, for this exact reason. She didn’t get a raise, but she didn’t get fired either. She got reprimanded, but her supervisor was confident that she would never make the same dumb mistake again. And now her story is used as a cautionary tale to drive home the importance of following procedure when we’re training new hires.




  • AKA Schrödinger’s Backup. Until you have successfully restored from a backup, it is just an amorphous blob of data that may or may not be valid.

    I say this as someone who has had backups silently fail. For instance, just yesterday, I had a managed network switch generate an invalid config file for itself. I was making a change on the switch, and saved a backup of the existing settings before changing anything. That way I could easily reset the switch to default and push the old settings to it, if the changes I made broke things. And like an idiot, I didn’t think to validate the file (which is as simple as pushing the file back to the switch to see if it works) before I made any changes.

    Sure enough, the change I made broke something, so I performed a factory reset and went to upload that backup I had saved like 20 minutes prior… When I tried to restore settings after the factory reset, the switch couldn’t read the file that it had generated like 20 minutes earlier.

    So I was stuck manually restoring the switch’s settings, and what should have been a quick 2 minute “hold the reset button and push the settings file once it has rebooted” job turned into a 45 minute long game of “find the difference between these two photos” for every single page in the settings.


  • Yeah, heat dissipation is surprisingly difficult in space, because the only real way to do it is via radiation. And radiation is one of the least effective methods of dissipating heat.

    The vast majority of heat transfer on earth happens via physical contact, in the form of fluids or solids touching each other. That’s what a heat sink is for. It increases surface area, so more fluid (air) can touch it and carry heat away. But without some sort of fluid contact, a heat sink isn’t going to help much. It’ll act as a radiator, but the cooling efficiency will only be a fraction of what is achieved via traditional forced air cooling.


  • We’ve lived through an entire internet without age checks, why is it different now? There aren’t more creeps

    I think the big difference is ease of access. For millennials growing up, accessing the internet basically required being at the family desktop in the middle of the living room. Phones weren’t connected to the internet, and cell phones weren’t even common yet.

    And kids still got groomed, even when their only access to the internet was in a shared family space. And that began to get more prevalent as devices became smarter and more portable. Now, any 8 year old can get groomed in their own bedroom, while simply playing a video game.


  • I actually disagree, because hardware-level verification is basically the most privacy-conscious method of accurately verifying a user’s age. Rather than fighting age verification entirely, I think it’s more productive to start assuming users are under 18 until proven otherwise… Age verification is inevitable, (if you don’t like it, tor is always an option), so we should at least figure out secure and private ways of doing so. Rather than resisting it outright, present them with secure and safe ways to do it. The internet is a dark place full of a lot of creeps, and services like Roblox have proven that they will enthusiastically become nesting grounds for predators unless they’re forced to add safeguards.

    Sure, it’s easy to say “just monitor your kids” but no parent can be present 24/7. And in fact, oftentimes parents end up using screen time so they can do other things like chores, without needing to watch their kid. So the “just watch your kids” argument is diametrically opposed to the reality of why parents tend to rely on screens. Sometimes you just need 15 minutes to wash the dishes, without a kid demanding your constant attention. Even I, a child-free person, can understand that. And it becomes increasingly difficult to monitor them as they grow into teens and (reasonably) start expecting their own privacy.

    I’ve been saying for a while now that we need to shift to hardware verification. Your device (or for shared devices like desktops, your user account) verifies your age once. And then it doesn’t need to do so again. All of the various sites and apps can simply ask your device “hey, is this user over {age}?” And the device responds with a simple true/false. You’re not needing to give your PII to every single site you visit, and the device isn’t needing to report back to the government every time an age verification check happens. It’s all done locally. The handshake could even be cryptographically secured, to prevent tech-savvy kids from MITM’ing the age check. And then protecting kids online is as simple as not age-verifying their device (and protecting your own password on shared devices). Hell, devices like cell phones could even have the age bracket set by the parent directly, since the phone would be on the parent’s phone bill. Similarly, parents could create child accounts on their shared devices, so kids can access age-appropriate content. It won’t stop kids from getting a prepaid phone, but it’ll at least prevent them from easily verifying that phone.

    And it’s also the most elegant for the user experience. As far as the adult user is concerned, they never even see an “are you over 18” verification when they visit a porn site. They simply get access to the site. And kids simply get redirected back to Google’s home page (or more realistically, a page on the porn site saying “hey you failed the age check. If you’re over 18, be sure you do that with your device before trying again, because this is the only page you’ll be able to access until then. Or if you’re under 18, click here to return to where you were before” explanation) as soon as the age check fails.

    Hardware age verification is basically the best of every world. You don’t rely on a third-party service to verify your PII (which will inevitably leak it, like Discord did). You don’t need to verify with every single individual site and service. The government doesn’t get a record of every site that asks for verification. And kids are automatically prevented from stumbling across adult content.

    I agree that Colorado democrats are typically the “if we cozy up to the right they might stop being mean to us” candidates. I think this bill is a poor implementation, but it’s at least done under the right premise. If we could force hardware manufacturers and/or OSes to support native age verification, it would solve a lot of the current issues that we have.


  • Yeah, Tailscale’s “zero-config” idea is great as long as things actually work correctly… But you immediately run into issues when you need to configure things, because Tailscale locks you out of lots of important settings that would otherwise be accessible.

    For instance, the WiFi at my job blocks all outbound WireGuard connections. Meaning I can’t connect to my tailnet when I’m at work, unless I hop off the WiFi and tether to my personal cell phone (which has a monthly data cap). Tailscale is built on WireGuard, and WireGuard only. If I could swap it to use OpenVPN or IKEv2 instead, I could bypass the problem entirely. But instead, I’m forced to just run an OpenVPN server at home, and connect using that instead of using Tailscale.



  • Appeals aren’t an infinite thing. Each appeal goes to a higher court, and eventually will reach the SCOTUS. And at any point, the respective appellate court can refuse to accept the appeal, essentially saying that they agree with the lower court’s ruling and leaving it in effect.

    Each step of the appeals process basically asks if the lower court applied the corresponding laws correctly. And if they did, the appellate court looks at whether or not that law is constitutional. If both are true, (the law is constitutional and was applied correctly) then the appeal fails. Appeals are actually fairly hard to win, especially for laws that have lots of precedent. If a law already has lots of precedent and the lower court was simply applying the law the same way that other cases did, the appeal will almost certainly be shot down.

    That’s why lots of the big landmark “court strikes down law as unconstitutional” cases are from laws that were recently passed. There is no long-standing precedent for the recently passed law, so the lower courts have to set the precedent, and the appeal is actually what is deciding whether or not the law is constitutional.



  • Yes and no. The hardware companies have already said that they’re not interested in expanding production. They know it’s a bubble, and don’t want expanded production now to cause a glut in the future when the inevitable pop happens. So prices may not actually drop, (even after the pop), because the companies still won’t be producing more hardware than they currently are.

    My best guess is that we’ll have some dark data centers sitting around collecting dust, but the hardware they bought won’t actually flood the market and crash prices. If anything, since the US dollar’s value is essentially tied to Nvidia and OpenAI’s market share, a pop will only make the dollar less powerful and will counteract any potential drops in prices that may have otherwise happened. The companies will get a trillion dollar bailout when the pop happens, (because they’re too big to fail) then nothing will change about the current hardware prices.




  • My wife initially hated my piholes, because they broke some of her phone’s stuff. She runs stock Samsung Android, so lots of the built in Google stuff got broken. She was constantly complaining about it. We eventually spent an evening hunt-and-peck’ing the various blocked DNS requests, to see which ones were required for her phone to work properly, and which ones were just Google Adsense BS. Got her set up with a WireGuard VPN connection that automatically activates when she’s disconnected from the home WiFi, so she’s always protected.

    Now that she’s used to it, it’s like a wake up slap whenever she encounters ads. We moved a while ago, and all of my more advanced networking stuff (including the pihole) was sitting in a box until I had time to set it all up. She suddenly started seeing ads again, and was absolutely gobsmacked at how pervasive they are. What really sent her over the edge was when our Roku TV was paused, and went to its idle screen. The idle screen is an auto-scrolling image, and it had an ad plastered across the scrolling image. She was like “what the fuck we’re not even watching anything right now! It’s just idle! Why the hell are they advertising to us on the damned idle screen??” That was what finally pushed her to give me an evening to set all of the networking stuff back up.