dara

it's not funny anymore - seriously, don't use comcast business

This is long past being funny.

Dear @ComcastBusiness, regarding your latest blowing off our 3 days and counting of complete lack of IPv4 connectivity and your several closed tickets, the latest of which being CR020322088:

"I have verified that all of your Comcast internet services are working correctly."

Your gateway is moving ZERO IPv4 PACKETS. NONE. IPv6 moves fine. IPv4 does not.

I have told you this, repeatedly.

"I am able to login to your Comcast gateway and reach websites from within your gateway."

Yes, I know.

Do you know, @comcastbusiness, who told you that IPv4 could communicate with the gateway?

I DID.

I. TOLD. YOU.

The problem is IPv4 can't _cross_ the modem from the LAN.

"I am getting solid pings/signal from your Gateway Static IP and all signal levels are in the green."

Yes. I know. How do I know? I TOLD YOU ABOUT IT.

When I could log into the gateway.

Which, from the LAN side, I often can't. It refuses all logins.

I told you that, too.

"There is nothing I am finding that would cause the intermittency you are experiencing."

What we are experiencing, @ComcastBusiness, is COMPLETE FAILURE TO PASS IPv4 AT ALL TIMES.

Being able to pass IPv6, while cute, is not "intermittency." We have been down for THREE DAYS.

We have been down, @ComcastBusiness, for THREE DAYS, with ZERO PROGRESS ON YOUR SIDE.

This is the FOURTH MAJOR OUTAGE SINCE OCTOBER and you are STILL not LISTENING TO US.

"The best way to test your services would be to directly connect a single laptop or PC to the back of the Comcast modem... and run a speed test at http://speedtest.xfinity.com."

Hard to run an IPv4 speed test when you pass zero (0) IPv4 packets, don't you think?

"If you direct connect and are getting the correct speeds (100MBPS DOWNLOAD/15 MBPS UPLOAD) this means the issue could lie somewhere in your networking equipment or we have too many devices connected."

OUR SPEED IS ZERO.

YOU ARE PASSING NO IPv4 PACKETS FROM THE LAN SIDE.

NONE.

What's it take, @ComcastBusiness?

In the last three days you've told us our modem is broken but you won't send us a new one, that our modem is fine and that it's our fault (spoiler: it's not) and that we need to do speed test on a modem which passes NO PACKETS from the LAN.

And - and - @ComcastBusiness you have to understand - your side has managed to diagnose NONE OF THIS.

I told _you_ the modem can pass IPv4 from _itself_ to the WAN. I'm fully aware the modem can pass IPv4 from itself to the WAN!

The problem is it can't move them from the LAN.

We got the same runaround the previous 3 major outages since October too, btw, @ComcastBusiness - but this one's even more insulting because it is a _complete_ outage. The last time, we had _some_ service.

Now the only service we have is IPv6 and attacks on my blood pressure.

At this point, @ComcastBusiness, we're three (3) days into a complete outage and you're back to telling us nothing is wrong because some intern managed to get the modem to send an IPv4 ping.

Which I TOLD YOU IT COULD DO.

Here's me telling you, @ComcastBusiness, on Friday, that the modem could pass IPv6 traffic but not IPv4 from WAN to LAN, and that the modem itself could pass IPv4 traffic. Remember this? You confirmed it was added to the ticket!

Of course, you closed that ticket too.

"I would recommend that you start with basic troubleshooting, and restart all of your equipment or contact your IT professional."

I RAN A DIVISION-LEVEL MAIL SERVER ROOM AT THE MICROSOFT CORPORATION.

I AM THE LOCAL IT PROFESSIONAL.

YOU KNOW THIS. My god.

(It was the smallest microsoft division at the time, but nonetheless, it's still true.)

"Alternately we can schedule a technician visit to verify that your services are working properly but if it's beyond the modem the technician can charge a $99.95 service fee due to not being a Comcast issue."

@comcastbusiness why you gotta hurt me like this?

You told us last night @comcastbusiness that you could see the modem was malfunctioning and needed to be replaced.

You also told us you wouldn't send a technician and also wouldn't send us a replacement modem.

Is this @AGOWA time or what?

Are you ever going to move IPv4 packets again?

I need to know, @comcastbusiness. This is our 4th major outage since Oct, all of which you blamed on us but were your fault, & you won't even acknowledge a problem. We are 100% down for 3 full days and counting.

Is it @AGOWA time?

Because at this point, @ComcastBusiness, I kind of have to act on the idea that you're never moving another IPv4 packet again and we're just going to be down for... at least another week. Probably two.

Friends don't let friends do @ComcastBusiness.

Hey, @ZiplyFiber - are your sales offices open on Sundays? Because I got some answers from your business tech people last night without even having a business account and they were the right answers.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    enraged enraged
  • Tags
dara

lol okay

So I was doing some reading up on in-home COVID isolation protocols and one of the things I found was that a reasonable way to measure air exchange adequacy is to check carbon dioxide PPM counts, and open windows enough keep the CO2 count under 800ppm. So it's a pretty high number really, but - still, that's the number.

And that makes sense if you think about it, right? So I dug out my old greenhouse CO2 meter and set it up to see what we're looking at on the main floor - which is kind of serving as a buffer floor between the ground and upper floors as housemate isolates on the ground floor - and came up with a nice tight reading of... 526ppm... in a room with no open windows.

No nobody's really hanging out in that space so maybe it's gonna be kinda artificially low but that's still a really good number, so I went up to the smallish room where Anna and I have been hanging out all night and which has even more ventilation (one forced air register in, the door to the hallway out) checked that...

...and got a nice tight 620ish. In a small room with two people and no open windows, and one HVAC register.

And it's one thing to be pretty sure I run a nice tight HVAC ship, particularly given all our givens, but it's another thing to have some raw numbers that say yes, in fact, your fresh-air air exchange system is working really well, and this is one goddamn tight HVAC ship.

Arguably, it's too well. Once isolation times are over, I'll be closing it back up for heat efficiency, and now I have an air exchange target I can measure.

Brilliant.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    pleased pleased
  • Tags
dara

comcast business blows donkeys, leaves no donkey unsatisifed, part... 48, 49, and 50?

So predictably, Comcast Business Internet didn't call us as they'd scheduled, and at that point I was presuming we were fucked for at least the weekend, so I went back to trying to arrange a solution to drive our wired LAN by the shared access point on my phone.

This sounds hilariously doomed, and I described it as such on Twitter, but I'd been inching closer to pulling it off for a couple of days as I experimented with it, and then once I had my best possible shot I pulled the trigger...

...and it works. Holy hell it works. It's shaky, not gonna lie, and it's slower than even Comcast, and the latency is a bit of a mess, but... it works! We can even stream video using it, it's fast enough for that!

It's the fourth major outage since October, btw. All four their fault. But if I can get some percentage of our client machines back online, that's something, right?

Right?

We've shelled out a bunch of money to boost our wireless data quota, which sucks, but hey, if Comcast can't give us packets, we've got to get them somewhere. Our servers are all offline, but by god, we have printers!

(And also gaming and file sharing and shit. You know. Basics.)

Oh, the reason we know this latest one is also Comcast's fault is because Anna called and ripped 'em a new one while I was putting all this stuff together. Normally, I'm the bad cop... okay, I'm still the bad cop. That was Anna being the good cop, which tells you had mad they made her.

Anyway, I was listening to Anna talking to them, and found out that all our trouble tickets get keep getting closed because they've decided our modem is broken - probably correct - but they aren't willing to send us a new one to install ourselves and can't send a tech out here with a new one so we're just screwed.

Literally no one has told us this until now. Apparently the fault was detectable on Friday, so they knew even as they were telling me the modem was fine and it was our equipment's fault.

Being told that, by the way, is what got me to write the SICK OF YOUR SHIT ROLL CALL post because I damned well knew better. And apparently, so did they. But at least now we know why the problem tickets keep getting closed.

Phenomenal work. Absolutely phenomenal.

Anyway, we've been up a few hours now, and I absolutely can't believe my single-point shared-network phone to wired ethernet bullshit is staying up this long. It's like, the hard part is keeping it from exploding, and yet... so far... it's... well...

It's not actually stable?

But it's doing a pretty decent job maintaining the illusion of stability.

And I kind of respect that.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    tired worn out
  • Tags
dara

Another attack on a vaccination site

This is the natural result of @GOP pro-plague politics, and why we have the worst COVID-19 numbers in the world.
InMinivanHell - @inminivanhell · 9:08 AM · Jan 8, 2022
Thomas Apollo went to a mobile clinic in Tustin, CA to protest covid vaccines. After calling employees “murderers” he assaulted them. Punching numerous staff members, leaving one requiring emergency care, it took 7 officers and a stun gun to subdue him.

He was cited & released.

[EMBEDDED ABC7 VIDEO OF ATTACK (KABC-TV 7 Los Angeles)]
The @GOP decided a long time ago to operate on the philosophy of “our country or no country,” in that if they don’t get to rule, there’s no functional country _to_ rule, which means unending sabotage when out of power.

Talk all you want about how we could have 100000s fewer dead - millions fewer long-term ill - how we could have this under control with rational steps.

None of it matters when you have a literal party of plague and sabotage - the @GOP - intentionally making everything worse.

This is the reality you HAVE TO face when actually trying to _implement_ COVID response.

You can science all you want - I certainly do - but if you do so ignoring the political reality on the ground, including the fact that @FoxNews and the @GOP will sabotage at every turn...

...then your plans may as well be astrology for all the good they’ll do.

The way out of this is to make pro-plague politicians and pro-sabotage parties to go down hard. To lock them out of power, and make it stick, because that power is the _only_ consequence they care about.

It’s been so obvious - so outright stated - for so long, and yet, people _not_ on Team Plague keep ignoring this basic reality, acting, planning, setting their own expectations as if this defining political axiom doesn’t matter.

Or doesn’t exist.

Like antivaxxers do with COVID.

Things you don’t want to be real are _still real_, no matter how much you wish or pray or demand they weren’t, be it COVID or structural white supremacy or the fact that the [profile] gop
actively boost the plague via sabotage.

Until you actually accept that, you can’t fix _anything_.

And that refusal to acknowledge reality… that, maybe more than anything else, is this moment’s actual defining axiom.

Not that we want to change reality. That’s fine.

That we won’t even acknowledge what our current reality even is.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

dara

why are all the photos gone

The photos are gone because fucking Comcast Business keeps taking us offline and can't get us online again for days every time they do it.

The photos will be back when Comcast Business takes its boot off our connectivity's neck again.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    pissed off pissed off
  • Tags
dara

comcast business blows donkeys, leaves no donkey unsatisifed, part... 46?

After a couple of hours of trying, I finally got Tier 1 support to add the current situation to the current ticket. I did have to change the ticket number because they closed the original one without telling us, so that was great. And then they opened two others, one for tier 1 and tier 2. But in theory, this is on the tier 2 ticket now. Yay!

That's nearly two hours to get this text added to a ticket though. Jesus.

-----

The modem is not passing IPv4 packets across LAN to WAN. It _may_ be able to send IPv4 packets from _itself_ to the WAN, and it _can_ pass IPv4 packets from itself to the LAN, but it DOES NOT pass IPv4 packets _across_ LAN to WAN or (as far as I can tell) vice versa.

A simple test is using my laptop connected by wire directly to the modem, configured by the modem's DHCP. If I attempt to query your nameserver via IPv4 from this laptop, it _always_ fails. If I attempt to query your nameserver via IPv6 from this same laptop, it _always_ succeeds. See attached screencap.

The reason I believe the modem can talk IPv4 over the WAN is because if I use the built-in IPv4 ping functionality via the web interface, it reports success. Since it does so without details, I cannot be sure it's actually succeeding, as the administrative software is unreliable. But I suspect it's working.

However, any otherwise-identical attempt from the LAN side of the modem to use IPv4 pings to the same servers fail, 100% of the time.

The modem is showing other signs of IPv4 routing irregularities. I will describe one now:

If a laptop is connected to the LAN side of the modem with modem-issued DHCP address, pinging our fixed-IP machines on the same (LAN) side of the modem will sometimes work normally, sometimes succeed with great delay in ping issuance but _not_ response time, sometimes succeed with routing error complaints on some but not necessarily all packets, and sometimes (but rarely) fail outright, all within a few minutes of each other with no configuration changes.

The modem is showing other signs of irregular behaviour as well. I will describe the main one now:

The modem is sometimes refusing logins when given valid login credentials. If your support team resets the password to default, the default password will also not work, regardless of the number of resets.

I have discovered today that if one is attempting to login via the fixed IP (173.160.243.46) and it does not work, it will _probably_ work if one does exactly the same login attempt on the DHCP root address (10.1.10.1).

The same is also true in reverse. If login via the DHCP root address (10.1.10.1) is failing, trying exactly the same login via the fixed IP (173.160.243.46) will _probably_ work.

If there is a reasonable explanation for this behaviour other than modem failure, I am not seeing it.

For these and other reasons, I am strongly suspecting that the modem is not taking provisioning properly, regardless of what it is reporting. The only reprovisioning from your side that I have seen have any effect at all is a full factory reset from remote. (My attempt to do a factory reset via the front panel failed - as in, didn't seem to work at all - and as this was when it was refusing all logins, I could not try via the customer administrative access panels.)

At this point we are down for just short of 24 hours and we yet again request escalation to Tier 2. Tier 1 have demonstrated yet again that they are NOT CAPABLE of solving this problem, and this is our fourth major network outage caused by your side since October, all previous lasting between 2 and 5 days, this one so far only 1 day.

The ticket number, again, is [deleted]. Please add all of the above to the ticket. Thank you.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    pissed off pissed off
  • Tags
dara

comcast business blows donkeys, leaves no donkey unsatisifed, part... 43?

Yeah. I tagged 'em. At this point, why the hell not?

SICK OF YOUR SHIT ROLL CALL:

1. "This isn't on our side, you should talk to your local IT about this."

A: It's been on your side EVERY TIME. Eventually, Tier 2 fixes it.

B: I AM THE LOCAL IT. I ran the WGA division server room at Microsoft. I HAVE SOME IDEA WHAT I'M DOING.

2. "Have you power-cycled your modem?"

Classic.

3. "Are you sure you have the right password?"

You reset it to factory default. Three times. It won't start magically working.

4. "Windows should be restarted."

My fleet of Linux servers glares at you angrily.

5. "I'll re-provision the modem. Wait five minutes and try again."

That didn't work the last four times, why is it going to work now? Oh look, it didn't.

6. "Someone will call you Monday."

[No call]

"Tuesday"

[Calls wrong number]

"Wednesday"

[Tier 2 shows up and fixes it]

(If we're lucky. I think the quickest we've been back up was two and a half days.)

7. Me: "[X] in Tier 2 told us you need to do exactly this, and can do that."

Tier 1: "We can't do that."

[tries again]

Tier 1: "All done!"

[nothing actually done]

8. [support wakes up from idle after hours of silence and apparent inactivity] “Is it fixed yet?”

No. No, as you have apparently done nothing, it is not fixed yet. The problem will not magically go away. It. Is. Not. Fixed. Yet.

Honestly, and I mean this in all sincerity, it has felt _many times_ like we've been being trolled.

Tier 2 is generally good to great and totally know what they're doing. Nice people.

Tier 1 - by phone and online - was not always a nightmare. I remember this. But it is one now.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

dara

the adventures of phone trees and airhorns

Just tried to talk to Earthlink, in case they have fibre in our area yet and the evaluation services aren't aware of it.

After working my way through a _particularly_ nasty phone tree - seriously, who thinks fake typing sounds are a good idea - I reached someone.

It sounded like a Russian phishing sweatshop in there, for reals. I suppressed my laughter.

Ivan - or whoever - had to look up whether they offered fixed IP services on their business services, and came back after a reasonable period to confirm they did. So I gave him name and address and general requirements so he could look up our service location.

And he did some, and I was on hold for a couple of minutes, and he came back and said he would have to transfer me to his "Business people."

Except he answered on the "Business service" phone tree himself, and identified as that, so... that's fun. But I say okay.

Then I get a bit of hold music, and then SHARP METALLIC HORN NOISE RIGHT IN MY EAR, then more hold music, then a partial play of an recording that used the phrase "invalid transfer," and then I was abruptly disconnected.

So that went well.

We have all kinds of choices. As long as they all suck.

And again: this is the _sales_ experience.

Their support rating is _terrible_. I bet I can guess why.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

dara

comcast business blows donkeys, leaves no donkey unsatisifed, part... 45?

We're down again, it's Comcast's fault again, as usual Tier 1 service bots are presuming we're idiots and it's our fault. So I wrote this on Twitter.



I got into an argument a couple of months ago with a guy who was Damned Well Sure that I had a zillion internet options that could meet our needs other than Comcast, and that he - in Montana, I think? - damned well knew that and I should STFU about infrastructure internet.

As I sit in my fourth major provider-caused outage since October, knowing we can't get an on-site tech from our provider for at least four days, having no idea if or when we might hear from Tier 2 to be sure we actually need one, I think on that, while talking again to providers.

We have Comcast, who we're with now.

We have HughesNet Satellite. 25mbps maximum down - about 15% of what we have - and negligible speeds up.

We have ViaSat Satellite. 100mbps maximum down - a mere 50% cut - and negligible speeds up.

We have Earthlink! It's DSL! But 75mbps. 43% of our current, though I have to say, infinitely more than the 0% we have right now.

But what I know _from experience_ is that the copper in this neighbourhood is nightmarishly bad, as in dropped calls on landlines bad, so hard pass.

We have a variety of cell phone companies, of course. That's how I'm online right now. Bandwidth isn't bad, but hoo boy the latency. I don't know if they offer fixed IP these days, but I guess I should check.

And we have Ziply. They're fibre. The numbers are fine and I go to the website and find that these days they're offering fixed IP services at last! Are we in business?

I look for pricing, and can't find it, so I use their chat to ask, and they can't tell me! Because the business services page links to consumer sales chat, not business sales chat, and they give me a phone number.

It's the wrong phone number.

So I go back to the website and find the right phone number, and I call it, and after some phone tree, I actually get to business sales. Yay!

And they tell me there's a greater than 10 minute wait to talk to BUSINESS SALES and I should use the website or call back later.

Think about that for half a second.

It's a >10 minute wait and I should call back if I want them to... _sell me service_.

And I just think about what that implies for support.

Mind you, I've been trying for an hour to get online @ComcastBusiness support to confirm that they've received the information I provided them and added to the latest outage ticket, so it's hard to imagine it being too much worse.

It's literally a matter of "Please confirm the above DMs describing our current situation have been added to ticket [number]."

It's been an hour. I'm resending the request every 10 minutes.

To be fair, they said they needed the account name and address too about 20 minutes in.

But I gave them that.

No response.

So, yeah. As I sit here trying to get support even to acknowledge the information I've sent, and as I sit contemplating being told to call back to be able to talk to _sales_, I think about how many, many choices we have.

As long as we choose for everything to suck.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.

  • Current Mood
    pissed off pissed off
  • Tags
dara

a simple prototype

Right then. A very simple prototype air-splitter and filter, with the filter on one output only.





It's as basic as it looks. It's a plastic-lined cardboard box with an intake and two outputs, one of which leads outside, the other of which leads to a small, easily acquired, cheaply available HEPA filter with charcoal prefilter.

I ran a very small test load of two bath towels in the washing machine, then in the dryer, to see whether we'd get much of anything out of it, and we do. Air coming out the filter is at 125F - very warm, but not hot enough to scald. Even on such a small, short test load, it bumped relative humidity at the sensor outside the laundry room by three points, and temperature up by about half a degree C. This all despite the window being open for additional ventilation while we wait out a COVID isolation protocol.

Obviously, this minimalist prototype effort lacks any sort of shutter system, but for the conditions of the test that's fine. This is in no small part about discovering how much this kind of maximalist filter slows exhaust into the room/heat and moisture reclamation, and how often it ends up needing to be cleaned.

I do think HEPA is over the top here, but it was convenient and nicely sized for the application. Also, for those who would have reason for HEPA-type filtration, it'd good to find that it does in fact allow for meaningful airflow, at least so far, with the relatively low pressure of an electric dryer. I will try this with the charcoal pre-filter, then, depending upon outcomes, replace the charcoal with a simpler plastic mesh (much akin to a typical lint screen) and see how that affects things.

There is no extra occlusion to the output vent hose, though that's certainly something which could be done. Basically, I don't want to create out-of-spec backpressure for the dryer, and in particular, I don't want to occlude outbound airflow in the event of complete filter blockage. This approach means that even were the filter to become completely blocked the dryer would behave normally, which means it can be completely blocked come warmer weather without having to resort to disassembly.

At the moment, my anticipated benefit case for this is mostly in very cold and/or dry weather, when inside humidity drops into the low to mid 30s. While it is in no way a constant supply like a humidifier, I think it could help delay the drying out of the building as a whole, via humidity retention in various humidity mass objects like couches, carpets, and so on.

Also posted to ソ-ラ-バ-ド-のおん; comment count unavailable comments at Dreamwidth. Please comment there.