First, let's recall what Azure DevOps agents are. When you run your pipeline, it runs not "in the cloud", but on a specific machine. This machine can be either managed by Microsoft (Microsoft hosted agents) or in your network, both in the cloud and locally. These machines are called agents.

In fact, all the work is done by a special software that needs to be installed on your machine. This software connects to Azure DevOps and when the pipeline is launched, it executes jobs and tasks.

It is important to highlight that the software that connects to Azure DevOps, not the opposite way. This is done for understandable reasons. It allows you not to open unnecessary ports on your machines, which increases security.

So, in my case, for some unknown reason, this software lost the ability to connect to Azure DevOps. Based on the logs, I understood that there is a problem with connecting to the endpoint visualstudio.com/_apis/connectionData, It was interesting, that it was possible to connect to this endpoint from the browser without any problems.

Something blocked the outgoing connection. I checked everything: local firewall, NSG settings (it was a machine in Azure), internet access, etc. I even checked the proxy settings. Everything was fine.

I had to call on the help of colleagues - network engineers. We checked together the traffic and found out that the Agent software that was sending the request to the wrong proxy address. It turned out that not so far ago, the IP address of the proxy server for Internet access from the internal network was changed and the corresponding settings were updated in the Windows configuration. But for some reason, the Agent software continues to use the old proxy.

But at least now it was clear where to dig. So… When you add an agent, Azure DevOps provides you a PowerShell script that will do almost everything for you.

This script works fine with a normal network configuration but stops working when using a proxy. Why? Because the software of our agent does not know how to pull the proxy settings from the operating system automatically and he needs to manually specify the necessary parameters. This is done using the --proxyurl parameter for the config.cmd of the agent software, which is called at the end of the PowerShell script.

You can read more about proxy configuration for agents here.

It seems that this is it - a happy ending to the story, but it was not there. While I was dealing with the problem, I decided to reinstall the agent software. To do this, you need to delete the old one and install the new one with a powershell script. Moreover, in general, there can be several agents on the machine that will perfectly work in parallel with each other.
How to remove an agent is written here. To do this, use the .\config remove command

But the problem is that in order to completely remove the agent with this command, you need to have an active Personal Access Token for an account that has the right to add agents to AzureDevOps. The token with which the agent was created was expiring a long time ago, and my account did not have enough rights. Moreover, from the Azure DevOps side, the agent can be easily removed, but it is simply impossible to correctly deactivate the agent software without the appropriate PAT. You can only stop the corresponding Windows service and make it not start again.

In general, in the end, I had to raise a devops who had the appropriate rights to add agents and ask him to make a temporary PAT, with which I managed to completely remove the old agent software and install a new one with the correct proxy server configuration.

That's what we need to learn all the time. You will never know where you will get an issue…