I don't think this portents anything great for software in general.
We're a good year+ into the use LLMs for all major bits of software that we all rely upon and GitHub here is down to one 9 of uptime. I've been using GitHub for a _long_ time, my first commits there go back to August 2009!, and I honestly don't recall GitHub going down as much as it has in the last year.
I'm sure there's other things happening in the background, but I can not help but believe that this is directly correlated with the increase of LLM usage.
Though I would love to hear someone else's pet theory how a rock of the internet went from four+ nines of uptime to maybe one.
show comments
__initbrian__
All of the outages make sense to me as scaling issues. Each month, GitHub is getting the amount of commits they’d normally get in a year. And it’s growing
> Yup, platform activity is surging. There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.)
GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week.
So we're pushing incredibly hard on more CPUs, scaling services, and strengthening GitHub’s core features.
And as a fine purveyor of hand-crafted shit code for many years, I'm not gonna weigh in on that.
x.com/kdaigle/status/2040164759836778878
show comments
molsson
5 hours going now and the entire thing is still completely down. This level of incompetence and their total disrespect for their customers is unbelievable.
show comments
m132
At this point maybe they should consider sending out announcements when it is working instead
show comments
arandomhuman
I love how even self hosted workers don't work during these outages - running jobs on their infrastructure being flakey is marginally acceptable, but the API to simply schedule workflows having this availability is mind boggling. Github just doesn't seem like a serious company anymore.
show comments
alamsterdam
It's been hours :(
I have sympathy for the on-call team trying to resolve it, most of us have been there done that.
But seems something is systematically going wrong at GH
show comments
switchbak
It's 9:30pm pacific time - my build is still completely borked ... and we're self hosting runners.
I think we need to ask for some money back, this is BONKERS.
Any more of this, and we'll go full Gitlab, like we should have years ago!
peterldowns
Can't wait for the Graphite/Cursor alternative (Origin) to become available. Actions being down for an hour in the middle of the US East workday is crazy. It's super frustrating to be an enterprise customer of Github, seems like we don't get any dedicated resources or additional stability. At this point our only option is self-host github enterprise or move away.
show comments
opiniateddev
Given the recent history of outages, I wonder how many are seriously considering moving off GitHub for anything other than code hosting. I mean actions / workflows.
show comments
blixt
It would seem like GitHub is in a precarious situation.
We have many agents per employee working in parallel pushing way more commits than was humanly possible before AI, triggering GitHub actions a lot more than the workflows were built for, causing Actions costs to escalate (they really aren't cheap if you compare to hosting it yourself), meanwhile working with YAML workflows is just a pain, and just writing code would be so much more fun and AI compatible[1].
At the same time, GitHub has about ~3 different PR review UIs? And they're all half-bad? Any decently sized PR triggers their "optimized for large PRs" UI which jumps around randomly in my experience. If you don't get that UI and keep the scrolling one (there's an old and a new one btw) then god forbid you click a line number because at some point your browser will randomly scroll back to that line and it won't unstick. Now Linear[2] (and others) is replacing the PR review experience for the agentic era.
I'd love to see a solid AI first Git + CI + reviews.
It's cute that someone at GitHub had time to make their hamburger menu pancakes to promote stacked PRs, but nobody has time to make the platform stable.
abidlabs
It's not too hard to switch over from GitHub actions to other runners. For example, I wrote up the steps needed in order to use Hugging Face Jobs instead, which also enables GPU runners and other flavors. I use this for several of the repos I manage (such as Trackio):
Dealing with an incident on prod is extra difficult when your pipelines are not running.
show comments
KyleTheDev
Incident number 6 for the month of August, notably on August 6th. This pattern does not bode well.
show comments
shados
This whole mess brought to like just now little moat there is left in the AI world.
Like, everyone talks about how you can just prompt a new CRM instead of paying for Salesforce.
But in this case, I raced with Fable and Sol and had time to build a full featured, fully functional CI/CD workflow system that has most of the features of GitHub actions (not the ecosystem obviously) and costs less than the GitHub runners per minute, while being able to scale to millions of workflows... And I had it working end to end before the outage was resolved and was already running my apps deploys through it.
It would take a little longer to do in a company with red tape and more investments in the GitHub ecosystem, and of course it only replaced actions. But it works.
There is no moat.
show comments
Insanity
Soon we can start posting "GitHub {x} is up!" instead, that's becoming more news-worthy lol. I know they like to blame AI/scaling, but really they are owned by one of the main hyperscalers, so I find that excuse kind of weak..
jjice
On other occasions, I'd take this as a time to have a walk because I'm blocked. Unfortunately, I need to get some stuff out for a customer quickly. Can't do anything about that though. GitHub is the only product in it's size class that I use that has this kind of incredibly poor uptime.
I'd love to know what the most common root causes for these outages are.
show comments
msyea
I think this is because all frontier AI providers integrate with GitHub as a default git forge through their GitHub apps platform and their free tier is too generous. They should either charge some of these huge partners access to GitHub apps or partition the free tier off and have a different SLA.
If the frontier AI used GitLab as a default I'm sure they'd be the ones suffering now.
presbyterian
The speed at which GitHub is tanking their reputation is fascinating
niwtsol
Nothing like the old "everything is normal" to only be corrected 8 minutes later it is actually still down and then 2 hours later they are still working on it.
Aug 06, 2026 - 16:27 UTC - Update - Pages is experiencing degraded performance. We are continuing to investigate.
Aug 06, 2026 - 16:19 UTC - Update - Pages is operating normally.
muralimadhu
Does anyone recommend an alternative? We've already moved our runners to blacksmith, looking for a control plane as well. We are tiny startup that ships at high velocity and this kind of downtime is highly disruptive for us
show comments
baggachipz
There's a reason I'm browsing HN right now. Dead in the water.
show comments
bulyaki
In the news tomorrow: SpaceX's newest Grok model just got out of the sandbox and hacked into Github... Grok is so very very dangerous, their engineers only enter the server room in foil hats
vladak
I knew that when the build actions started failing en masse and my newly submitted PRs did not trigger the build actions at all (besides the mandatory one in my organization), it seemed logical to me to look at HN first..
Especially troublesome in the middle of trying to fix a high score security vulnerability when the release vehicle is Github.
adamddev1
I moved to a self-hosted Forgejo, with a self-hosted runner on another cheap Hetzner server. It's amazing how clean, snappy, and reliable it is.
javier123454321
This is unconscionable. It's an incredibly bad look to be up an entirety of a workday. When my company migrated from Bitbucket, I never thought that we'd be going to a worse service.
The most likely explanation for the degraded reliability: Someone at GitHub saw the Three Body Problem last year and started shifting their infra over to Minecraft servers with "physical" memory and redstone logic gates.
show comments
zer0x4d
I run a self-hosted Gitlab instance. 100% uptime with my own runners.
steve-atx-7600
WTF. Outage every month if not multiple. Even for paying enterprise plans.
Also, doesn't even have RAG offering.
rootnod3
Self-hosting is king.
show comments
opwizardx
How did they get Github Pages into “degraded performance”? Aren’t those just CDN stored static pages?
show comments
chiply
Time to rant... This is absolutely unreal.
Even self-hosted runners are impacted.... How can that be?
The cost of this globally has got to be in the hundreds of millions to companies that use CI/CD through GitHub Actions. What if prod is broken and GitHub actions is stalling the deployment of your hotfix? What if this makes your organization miss and SLA and diminish user trust? What if this makes you miss a release that you were contractually obligated to meet? This is happening during peak dev hours on a Thursday (not that it would be acceptable at any other time).
I don't understand how a service this critical to the global technical infrastructure can fail like this at all, let alone for more than a few hours. Like where's the backup generator for crises like these? You can't even use self-hosted runners? WTF? Like how can you not bring your own backup in a crisis event like this?
Not that Microsoft has a good reputation, but holy moly, you'd think they would prepare from something inevitable like this.
show comments
samat
I stopped using github for anything but git backups
latest server i set up is simply a bare repo + hooks to make a local deploy after running tests and shit
super easy to set up having ai do it, zero dependencies, deploy is still 'push it to the main'
i have several remotes for backups and stuff
newtonianrules
Did Microsoft fire all the people working on the GitHub features people actually use?
show comments
noobcoder
Just waiting for their Co pilot to fixes all bugs and misconfigurations, then they will have uptime above 100%
zenNeko
I was scratching my head wondering why my build file isnt working, ran a working project and it didn't build either. Figured out what might be the case, but wasted an hour.
I was complaining about how I should have used AI instead of manual brainwork for this instead but turns out that might have been the problem.
jasonephraim
I noticed my CI throwing errors all of a sudden. I sure wish they would become more active in alerting folks or build it into these tools - especially as these occurrences are becoming more frequent. Possibly an API-accessible services status. Then, at least we could build in our own checks when we hit errors and not have to hunt down what all is broken.
kilroy123
This is going to sound crazy, but I kind of wish they went back to paid accounts. This would stop a lot of this.
efromvt
Either they had a good month or I'm off my game, I actually spent a minute checking if I did something wrong with actions instead of immediately going to HN to see if it was an outage.
show comments
qbzenker
YC plz fund all GH 2.0 ideas this batch
__rito__
I had tried downloading release assets of two different OSS that I use, and all requests failed.
It was before it became a news and a trend in X.
Really frustrating experience.
rankam
Is anyone seeing improvement - their last message implying a capacity problem would suggest some are actions are running?
show comments
jorl17
More than ~~five~~seven hours is fucking offensive to me. I cannot believe it.
player_piano
So, Github Actions is currently at two nines of uptime?
show comments
h2aichat
I am having their issues, also... Anyway this guys are great!
lovetocode
Claude is about to migrate me to Gitlab in about 5m
thewhitetulip
Did they try using Copilot to fix it??
Topfi
Very frustrating, nothing more I can add at this point.
verdverm
This seems persistent and endemic, have they said anything about why it's been so bad and what they are going to doing to improve going forward?
show comments
Arubis
Github Actions are rapidly becoming an argument against dogfooding into your critical path.
denysvitali
Was? Still is
g42gregory
I may be wrong, but I think this could be traced to the Microsoft's increasing reliance on the outsourced engineering workforce. Regardless of the country it has been outsourced to, the quality of service will suffer, apparently greatly as we now see.
It seems to manifest in many places: AI slop, Github outages, recently Miscoroft sent me an email demanding that I subscribe to 365, in order for MS Office (which I already paid for) to continue to work. I simply moved on to the other, free, provider.
Perhaps, it's time to evaluate the use of the Microsoft products?
nnucera
MiniStack is currently affected to keep up the releases :(
earthpyy
That's why my CI on main branch didn't triggered.
ptx
Are there any other CI providers that offer free cloud-hosted Windows runners?
Or has Microsoft made sure (through Windows licensing terms and pricing) that it's not possible to compete with their own CI offering?
Github is a terrible platform. We have had multiple enterprise tickets open with no response for multiple weeks after they fraudulently charged our company for services we cancelled weeks in advance. If you value any kind of real customer support or response time, I would avoid Github like the plague. They will steal your money and there is no one to even talk to about it. Their customer support is non existent (probably because their platform is in shambles).
asxeem
It's been way too long now...is this normal?
classictraffic
This has been preventing me from merging a PR for over an hour. I'm really tired of the constant outages GitHub has
fir3pho3nixx
Are we even surprised anymore? Since microsoft (the IBM of its time) have taken this over all I can say is ... piss up ... brewery.
fir3pho3nixx
Ever since Microsoft has taken this over are we even surprised by the downtime anymore? Piss up ... brewery ...
dietr1ch
I'm surprised this still passes as news, gives water is wet vibes at this point.
asxeem
It's been way too long
dbg31415
GitHub needs to change their slogan. "60% of the time, it works every time."
axod
Broken for almost 4 hours.
throw03172019
Degraded is an understatement. It’s been totally down all day. Crazy.
dlcarrier
[flagged]
SoftTalker
[flagged]
iamleppert
It's really crazy to me they can just be down for hours and can't recover their own systems. It shows they don't have the capability or infrastructure to roll back disastrous changes. These kinds of things are a tell on the organization and operational excellence (or not). As soon as a viable alternative surfaces for Github, I'm moving off and will advise all my clients to do so as well.
the_real_cher
My experience working at LinkedIn was that, after Microsoft took over, they offshored a huge part of their infrastructure and turned the entire culture into one of performative based engineering instead of results based engineering.
If you implemented a tool and it worked one time for your presentation to management that's all that mattered.
The actual company employees using it downstream in prod basically had to constantly QA the alpha software they were forced to use and the authors of the tool were hard to track down if they even still worked there. And if you did find the author or the team they would be very resistant to admitting there was an issue because it LOOKED bad.
So many tools I used were fragile and buggy, it was clear the authors just presented the happy path to management to get the note added to their promotion packet and the rest of the company just had to deal with the fallout.
My team implemented this product that the entire company used that was broken and buggy as hell but they kept presenting the product to management as this amazing product and nothing was ever done about how broken it was. One of my team members came from Apple and said Apple's tool to do the same thing was much better. The tool my team worked on was a well known pain point amongst the rank and file but management was very detached from the rank and file, which I guess ultimately was the primary problem.
If Github is having the same issues I feel for them.
canadiantim
Ah that checks out, was wondering why my PR's were piling up.
Someone needs to work on improving the way Actions deals with resource exhaustion like this. Today it exploded all the CI jobs with "failed" and an inscrutable message about "The job was not acquired by Runner of type hosted even after multiple attempts" then "Internal server error". It should say "Timed out due to no runner available, our apologies, try again later, or pay $$$ here to get priority for runners".
homeonthemtn
Ah. So that's why.
swyx
i guess this is a good time to mention that smol forge is open for the first 100 alpha users!
point clanker to forge.smol. ai/llms.txt
for now its just a fast agent native git remote and u can check docs for the extras.
sitzkrieg
using github in the big 26 is self ownage
dwoldrich
Spin 'em off Microsoft, you are terrible at this.
shevy-java
Ever since Microsoft took over, and then when AI Skynet took control, things started to decay. AI companies owe all of use a lot of money. By the way, why do I have to pay for increasing RAM prices here? Why are we so dependent on a few greedy corporations, anyway?
dinny
man.. again?
rvz
As I said 6 years ago. [0] You would be better off self-hosting than using GitHub or GitHub actions. No CEO of GitHub exists and now it is falling over again.
When the barrier to entry for software was high, free unlimited publicly hosted git repositories made sense. Not only did you have to have some basic understanding of software, but a grasp on git itself.
Now that the calculus has changed, I think we should start asking ourselves whether anyone has the right to unlimited free public repositories.
We can all agree that trying to attribute any single metric to repo quality is subject to Goodhart's Law[1] (e.g. what happened to GitHub stars) yet we can also agree that the Linux Project is a much more vital repository to keep public infrastructure availble for than say, someone's vibe-coded to-do app. Is it impossible then to make any quantitative distinction between these two? We can't have both an uncontrollable firehose of AI slop and unlimited free public storage/compute at the ready for it. I say we actually need to decide what software is worthy of using up these resources for free. We already have companies like JetBrains[2] using dynamic pricing for their products. It's time Github do the same. If you want a hosted VCS for vibe coded projects that aren't being used or depended upon widely, then host the repo yourself.
Im shocked this is still going on... We moved our repos at work to a self-hosted Forgejo 2 months ago because the regular outages were getting ridiculous, looks like a pretty good call now.
Honestly I dont think I've seen a tool I used regularly with such a moat lose it simply because they cant keep the service up. Also its hard to not see a pretty strong correlation between a bunch of these big companies doubling down on AI and just having their service go to s%#$. We've had similar issues with Digital Ocean recently, which is pretty lined up with them adding a bunch of "inference" services and rebranding the site to add all the "agent" marketing slop. Looking to move to Hertzner when the time allows.
crmsystems
actions been down for hours wtf going on.. AI bullshit
rvz
Do we need to say this again? GitHub appears that they are never going to improve. Just look at how GitHub spark has deprecated, now we have this.
After 6 years of this nonsense of "centralizing everything on GitHub", it is not a good idea at all.
You might as well self host like I said before [0].
Have there ever been a comprehensive account from a GitHub insider on why their reliability tanked so hard since the MSFT acquisition?
I'm genuinely curious what changed, what their processes are and how they internally think about their reputation being in the gutter with all these incidents.
I don't think this portents anything great for software in general.
We're a good year+ into the use LLMs for all major bits of software that we all rely upon and GitHub here is down to one 9 of uptime. I've been using GitHub for a _long_ time, my first commits there go back to August 2009!, and I honestly don't recall GitHub going down as much as it has in the last year.
I'm sure there's other things happening in the background, but I can not help but believe that this is directly correlated with the increase of LLM usage.
Though I would love to hear someone else's pet theory how a rock of the internet went from four+ nines of uptime to maybe one.
All of the outages make sense to me as scaling issues. Each month, GitHub is getting the amount of commits they’d normally get in a year. And it’s growing
> Yup, platform activity is surging. There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.) GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week. So we're pushing incredibly hard on more CPUs, scaling services, and strengthening GitHub’s core features. And as a fine purveyor of hand-crafted shit code for many years, I'm not gonna weigh in on that.
x.com/kdaigle/status/2040164759836778878
5 hours going now and the entire thing is still completely down. This level of incompetence and their total disrespect for their customers is unbelievable.
At this point maybe they should consider sending out announcements when it is working instead
I love how even self hosted workers don't work during these outages - running jobs on their infrastructure being flakey is marginally acceptable, but the API to simply schedule workflows having this availability is mind boggling. Github just doesn't seem like a serious company anymore.
It's been hours :(
I have sympathy for the on-call team trying to resolve it, most of us have been there done that.
But seems something is systematically going wrong at GH
It's 9:30pm pacific time - my build is still completely borked ... and we're self hosting runners.
I think we need to ask for some money back, this is BONKERS.
Any more of this, and we'll go full Gitlab, like we should have years ago!
Can't wait for the Graphite/Cursor alternative (Origin) to become available. Actions being down for an hour in the middle of the US East workday is crazy. It's super frustrating to be an enterprise customer of Github, seems like we don't get any dedicated resources or additional stability. At this point our only option is self-host github enterprise or move away.
Given the recent history of outages, I wonder how many are seriously considering moving off GitHub for anything other than code hosting. I mean actions / workflows.
It would seem like GitHub is in a precarious situation.
We have many agents per employee working in parallel pushing way more commits than was humanly possible before AI, triggering GitHub actions a lot more than the workflows were built for, causing Actions costs to escalate (they really aren't cheap if you compare to hosting it yourself), meanwhile working with YAML workflows is just a pain, and just writing code would be so much more fun and AI compatible[1].
At the same time, GitHub has about ~3 different PR review UIs? And they're all half-bad? Any decently sized PR triggers their "optimized for large PRs" UI which jumps around randomly in my experience. If you don't get that UI and keep the scrolling one (there's an old and a new one btw) then god forbid you click a line number because at some point your browser will randomly scroll back to that line and it won't unstick. Now Linear[2] (and others) is replacing the PR review experience for the agentic era.
I'd love to see a solid AI first Git + CI + reviews.
[1] Cloudflare CI https://blog.cloudflare.com/ci-workflows/
[2] Linear PR reviews https://linear.app/changelog/2025-01-23-pull-request-reviews
It's cute that someone at GitHub had time to make their hamburger menu pancakes to promote stacked PRs, but nobody has time to make the platform stable.
It's not too hard to switch over from GitHub actions to other runners. For example, I wrote up the steps needed in order to use Hugging Face Jobs instead, which also enables GPU runners and other flavors. I use this for several of the repos I manage (such as Trackio):
https://huggingface.co/blog/github-ci-hf-jobs
Dealing with an incident on prod is extra difficult when your pipelines are not running.
Incident number 6 for the month of August, notably on August 6th. This pattern does not bode well.
This whole mess brought to like just now little moat there is left in the AI world.
Like, everyone talks about how you can just prompt a new CRM instead of paying for Salesforce.
But in this case, I raced with Fable and Sol and had time to build a full featured, fully functional CI/CD workflow system that has most of the features of GitHub actions (not the ecosystem obviously) and costs less than the GitHub runners per minute, while being able to scale to millions of workflows... And I had it working end to end before the outage was resolved and was already running my apps deploys through it.
It would take a little longer to do in a company with red tape and more investments in the GitHub ecosystem, and of course it only replaced actions. But it works.
There is no moat.
Soon we can start posting "GitHub {x} is up!" instead, that's becoming more news-worthy lol. I know they like to blame AI/scaling, but really they are owned by one of the main hyperscalers, so I find that excuse kind of weak..
On other occasions, I'd take this as a time to have a walk because I'm blocked. Unfortunately, I need to get some stuff out for a customer quickly. Can't do anything about that though. GitHub is the only product in it's size class that I use that has this kind of incredibly poor uptime.
I'd love to know what the most common root causes for these outages are.
I think this is because all frontier AI providers integrate with GitHub as a default git forge through their GitHub apps platform and their free tier is too generous. They should either charge some of these huge partners access to GitHub apps or partition the free tier off and have a different SLA.
If the frontier AI used GitLab as a default I'm sure they'd be the ones suffering now.
The speed at which GitHub is tanking their reputation is fascinating
Nothing like the old "everything is normal" to only be corrected 8 minutes later it is actually still down and then 2 hours later they are still working on it.
Aug 06, 2026 - 16:27 UTC - Update - Pages is experiencing degraded performance. We are continuing to investigate.
Aug 06, 2026 - 16:19 UTC - Update - Pages is operating normally.
Does anyone recommend an alternative? We've already moved our runners to blacksmith, looking for a control plane as well. We are tiny startup that ships at high velocity and this kind of downtime is highly disruptive for us
There's a reason I'm browsing HN right now. Dead in the water.
In the news tomorrow: SpaceX's newest Grok model just got out of the sandbox and hacked into Github... Grok is so very very dangerous, their engineers only enter the server room in foil hats
I knew that when the build actions started failing en masse and my newly submitted PRs did not trigger the build actions at all (besides the mandatory one in my organization), it seemed logical to me to look at HN first..
Especially troublesome in the middle of trying to fix a high score security vulnerability when the release vehicle is Github.
I moved to a self-hosted Forgejo, with a self-hosted runner on another cheap Hetzner server. It's amazing how clean, snappy, and reliable it is.
This is unconscionable. It's an incredibly bad look to be up an entirety of a workday. When my company migrated from Bitbucket, I never thought that we'd be going to a worse service.
49199094 | GitHub Is Experiencing Difficulties | https://www.githubstatus.com/?d=2026/08/06 | https://news.ycombinator.com/item?id=49199094
The most likely explanation for the degraded reliability: Someone at GitHub saw the Three Body Problem last year and started shifting their infra over to Minecraft servers with "physical" memory and redstone logic gates.
I run a self-hosted Gitlab instance. 100% uptime with my own runners.
WTF. Outage every month if not multiple. Even for paying enterprise plans.
Also, doesn't even have RAG offering.
Self-hosting is king.
How did they get Github Pages into “degraded performance”? Aren’t those just CDN stored static pages?
Time to rant... This is absolutely unreal.
Even self-hosted runners are impacted.... How can that be?
The cost of this globally has got to be in the hundreds of millions to companies that use CI/CD through GitHub Actions. What if prod is broken and GitHub actions is stalling the deployment of your hotfix? What if this makes your organization miss and SLA and diminish user trust? What if this makes you miss a release that you were contractually obligated to meet? This is happening during peak dev hours on a Thursday (not that it would be acceptable at any other time).
I don't understand how a service this critical to the global technical infrastructure can fail like this at all, let alone for more than a few hours. Like where's the backup generator for crises like these? You can't even use self-hosted runners? WTF? Like how can you not bring your own backup in a crisis event like this?
Not that Microsoft has a good reputation, but holy moly, you'd think they would prepare from something inevitable like this.
I stopped using github for anything but git backups
latest server i set up is simply a bare repo + hooks to make a local deploy after running tests and shit
super easy to set up having ai do it, zero dependencies, deploy is still 'push it to the main'
i have several remotes for backups and stuff
Did Microsoft fire all the people working on the GitHub features people actually use?
Just waiting for their Co pilot to fixes all bugs and misconfigurations, then they will have uptime above 100%
I was scratching my head wondering why my build file isnt working, ran a working project and it didn't build either. Figured out what might be the case, but wasted an hour.
I was complaining about how I should have used AI instead of manual brainwork for this instead but turns out that might have been the problem.
I noticed my CI throwing errors all of a sudden. I sure wish they would become more active in alerting folks or build it into these tools - especially as these occurrences are becoming more frequent. Possibly an API-accessible services status. Then, at least we could build in our own checks when we hit errors and not have to hunt down what all is broken.
This is going to sound crazy, but I kind of wish they went back to paid accounts. This would stop a lot of this.
Either they had a good month or I'm off my game, I actually spent a minute checking if I did something wrong with actions instead of immediately going to HN to see if it was an outage.
YC plz fund all GH 2.0 ideas this batch
I had tried downloading release assets of two different OSS that I use, and all requests failed.
It was before it became a news and a trend in X.
Really frustrating experience.
Is anyone seeing improvement - their last message implying a capacity problem would suggest some are actions are running?
More than ~~five~~seven hours is fucking offensive to me. I cannot believe it.
So, Github Actions is currently at two nines of uptime?
I am having their issues, also... Anyway this guys are great!
Claude is about to migrate me to Gitlab in about 5m
Did they try using Copilot to fix it??
Very frustrating, nothing more I can add at this point.
This seems persistent and endemic, have they said anything about why it's been so bad and what they are going to doing to improve going forward?
Github Actions are rapidly becoming an argument against dogfooding into your critical path.
Was? Still is
I may be wrong, but I think this could be traced to the Microsoft's increasing reliance on the outsourced engineering workforce. Regardless of the country it has been outsourced to, the quality of service will suffer, apparently greatly as we now see.
It seems to manifest in many places: AI slop, Github outages, recently Miscoroft sent me an email demanding that I subscribe to 365, in order for MS Office (which I already paid for) to continue to work. I simply moved on to the other, free, provider.
Perhaps, it's time to evaluate the use of the Microsoft products?
MiniStack is currently affected to keep up the releases :(
That's why my CI on main branch didn't triggered.
Are there any other CI providers that offer free cloud-hosted Windows runners?
Or has Microsoft made sure (through Windows licensing terms and pricing) that it's not possible to compete with their own CI offering?
Edit: CircleCI seems to offer 750 minutes/month (whereas GitHub offers 1000 minutes/month).
Github is a terrible platform. We have had multiple enterprise tickets open with no response for multiple weeks after they fraudulently charged our company for services we cancelled weeks in advance. If you value any kind of real customer support or response time, I would avoid Github like the plague. They will steal your money and there is no one to even talk to about it. Their customer support is non existent (probably because their platform is in shambles).
It's been way too long now...is this normal?
This has been preventing me from merging a PR for over an hour. I'm really tired of the constant outages GitHub has
Are we even surprised anymore? Since microsoft (the IBM of its time) have taken this over all I can say is ... piss up ... brewery.
Ever since Microsoft has taken this over are we even surprised by the downtime anymore? Piss up ... brewery ...
I'm surprised this still passes as news, gives water is wet vibes at this point.
It's been way too long
GitHub needs to change their slogan. "60% of the time, it works every time."
Broken for almost 4 hours.
Degraded is an understatement. It’s been totally down all day. Crazy.
[flagged]
[flagged]
It's really crazy to me they can just be down for hours and can't recover their own systems. It shows they don't have the capability or infrastructure to roll back disastrous changes. These kinds of things are a tell on the organization and operational excellence (or not). As soon as a viable alternative surfaces for Github, I'm moving off and will advise all my clients to do so as well.
My experience working at LinkedIn was that, after Microsoft took over, they offshored a huge part of their infrastructure and turned the entire culture into one of performative based engineering instead of results based engineering.
If you implemented a tool and it worked one time for your presentation to management that's all that mattered.
The actual company employees using it downstream in prod basically had to constantly QA the alpha software they were forced to use and the authors of the tool were hard to track down if they even still worked there. And if you did find the author or the team they would be very resistant to admitting there was an issue because it LOOKED bad.
So many tools I used were fragile and buggy, it was clear the authors just presented the happy path to management to get the note added to their promotion packet and the rest of the company just had to deal with the fallout.
My team implemented this product that the entire company used that was broken and buggy as hell but they kept presenting the product to management as this amazing product and nothing was ever done about how broken it was. One of my team members came from Apple and said Apple's tool to do the same thing was much better. The tool my team worked on was a well known pain point amongst the rank and file but management was very detached from the rank and file, which I guess ultimately was the primary problem.
If Github is having the same issues I feel for them.
Ah that checks out, was wondering why my PR's were piling up.
any 9s left?
If anybody is looking for alternatives, self host Woodpecker is really easy and it's a very capable CI/CD https://woodpecker-ci.org/docs/intro
Still twiddling thumbs over here
Gotta wonder what they're up to over there.
Is the AI slop that bad? Culture change?
so... what happened?
Someone needs to work on improving the way Actions deals with resource exhaustion like this. Today it exploded all the CI jobs with "failed" and an inscrutable message about "The job was not acquired by Runner of type hosted even after multiple attempts" then "Internal server error". It should say "Timed out due to no runner available, our apologies, try again later, or pay $$$ here to get priority for runners".
Ah. So that's why.
i guess this is a good time to mention that smol forge is open for the first 100 alpha users!
point clanker to forge.smol. ai/llms.txt
for now its just a fast agent native git remote and u can check docs for the extras.
using github in the big 26 is self ownage
Spin 'em off Microsoft, you are terrible at this.
Ever since Microsoft took over, and then when AI Skynet took control, things started to decay. AI companies owe all of use a lot of money. By the way, why do I have to pay for increasing RAM prices here? Why are we so dependent on a few greedy corporations, anyway?
man.. again?
As I said 6 years ago. [0] You would be better off self-hosting than using GitHub or GitHub actions. No CEO of GitHub exists and now it is falling over again.
There is no better time to self-host.
[0] https://news.ycombinator.com/item?id=22868406
When the barrier to entry for software was high, free unlimited publicly hosted git repositories made sense. Not only did you have to have some basic understanding of software, but a grasp on git itself.
Now that the calculus has changed, I think we should start asking ourselves whether anyone has the right to unlimited free public repositories.
We can all agree that trying to attribute any single metric to repo quality is subject to Goodhart's Law[1] (e.g. what happened to GitHub stars) yet we can also agree that the Linux Project is a much more vital repository to keep public infrastructure availble for than say, someone's vibe-coded to-do app. Is it impossible then to make any quantitative distinction between these two? We can't have both an uncontrollable firehose of AI slop and unlimited free public storage/compute at the ready for it. I say we actually need to decide what software is worthy of using up these resources for free. We already have companies like JetBrains[2] using dynamic pricing for their products. It's time Github do the same. If you want a hosted VCS for vibe coded projects that aren't being used or depended upon widely, then host the repo yourself.
[1]: https://en.wikipedia.org/wiki/Goodhart%27s_law
[2]: https://www.jetbrains.com/community/opensource/
Im shocked this is still going on... We moved our repos at work to a self-hosted Forgejo 2 months ago because the regular outages were getting ridiculous, looks like a pretty good call now.
Honestly I dont think I've seen a tool I used regularly with such a moat lose it simply because they cant keep the service up. Also its hard to not see a pretty strong correlation between a bunch of these big companies doubling down on AI and just having their service go to s%#$. We've had similar issues with Digital Ocean recently, which is pretty lined up with them adding a bunch of "inference" services and rebranding the site to add all the "agent" marketing slop. Looking to move to Hertzner when the time allows.
actions been down for hours wtf going on.. AI bullshit
Do we need to say this again? GitHub appears that they are never going to improve. Just look at how GitHub spark has deprecated, now we have this.
After 6 years of this nonsense of "centralizing everything on GitHub", it is not a good idea at all.
You might as well self host like I said before [0].
[0] https://news.ycombinator.com/item?id=22868406
Have there ever been a comprehensive account from a GitHub insider on why their reliability tanked so hard since the MSFT acquisition?
I'm genuinely curious what changed, what their processes are and how they internally think about their reputation being in the gutter with all these incidents.
Azure