Prometheus: Percentage output with labels from source

I have been trying to work this out for myself and managed to get somewhere.
I have a golang application which extracts workitems from Azure DevOps and creates prometheus metrics.

The Problem

I have a metric called azdo_worktem_tag with labels of project and tag, I wanted to get a percentage of tags prefixed with the value of each unique value of “Team:” over all the tags prefixed with “Team:”.

azdo_workitem_tag{project="xyz" tag="Team: a"} 1
item_metric{project="xyz" tag="Team: b"} 1

In some past solutions I have been met with a lot of errors, or I have been able get something, but the output is labeled as “Value” which made it hard to fix the legends.

The Journey…

I started off with the initial sum by tag:

sum(azdo_workitem_tag{project="xyz",tag=~"Team:.*"}) by (tag)

In all the sites I found discussing this topic they were always mentioning about on and ignoring and group_left _right and a lot of concepts involving those, but I kept getting either a 0 result or an error.

In the end I came upon a stackoverflow answer that helped make sense.

The problem I was not getting my head around was the labels on each metric were crossing over each other and as mentioned in the answer you have to ignore the ones you are filtering.

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"})

The next error I got then made sense “multiple matches for labels: many-to-one matching must be explicit (group_left/group_right)”.

I have to be honest, I lucked out a bit on the next iteration, when I accidentally fired the query too soon! I set group_left and was about to hit shift + 9 but instead hit shift + return and ended up with a desired output.

The Solution

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) group_left 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"})

The output of this provides a percental value (which you then multiply bt 100) with a label of the tag.

By then adding project to the sum by, the metric returned was then also labeled with the project label.

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) group_left 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) 
* 100
visual from grafana explorer

On Reflection

This was a useful learning experience I have a few other situations where I need to use 2 metrics to help me work out some other facts and I think this will help as a good baseline.

PowerShell: Disconnecting Users

What often gets old is when people disconnect from servers and leave files and folders locked, that could affect a release run.

nb: This isnt 100% pure powershell, but I am using powershell to perform the loop behaviour needed

foreach($item in (query user /server:$env:COMPUTERNAME 2>&1 | select -skip 1))
{
    $sessionId = ($item -split "\s+")[-5]
    Write-Verbose "Logging Off Session: $sessionId"
    logoff $sessionId /server:$env:COMPUTERNAME
}

query user /server:[Machinename] queries for user sessions, both connected and disconnected.

$env:COMPUTERNAME targets the current machine, you could pass a known machine name to the query.

2>&1 downgrades errors to information

This deals with any session, so even those currently connected, which adds even more fun!

tools: I cannot spell or do grammar good!

I don’t think that I write that well, and I generally just type and type until something looks ok, then I have to go back and review what I have typed to make sure it makes sense, using a tool like Grammarly helps with that.

It doesn’t improve your writing but it does clearly show up any spelling mistakes or grammatical errors, the spelling mistakes will provide a well thought out list of options, and for me, it is clear enough that I can leave it and come back to it later. This I feel allows me to carry on with just typing out my ideas and then going back and reviewing the clear violations of the Engish language that I have just put on a page.

I often find myself right-clicking which is the default behaviour for the Chrome spell checker, but I am getting used to the left click actions they provide.

With WordPress with chrome extension

I have sometimes noticed that the choice I make isn’t always selected, so it sometimes takes an extra go to get it to apply the grammar police correctly. I also think it might be the refresh rate of the sites I use that don’t understand that I have changed something.

Other options

Grammarly also comes with some extras that you can add to office and word, chrome, and just as a stand alone desktop application.

Summary

It doesn’t make me write better, but I think it gives me more freedom to just type. I actually used it on this blog post!

PowerShell: Setting up a Register-ScheduledJob

Let’s look at something that I did, I needed to keep an eye on all the recycle bins on a server, that was a separate problem, but I wanted to schedule the job to occur once a week.

Just in case you are interested the script to do the recycle clean looks like this

$Shell = New-Object -ComObject Shell.Application
$Global:Recycler = $Shell.NameSpace(0xa)

$Global:Recycler.Items() | Remove-Item -Recurse

Register-ScheduledJob requires a ScriptBlock, so this becomes

$scriptBlock = {
$Shell = New-Object -ComObject Shell.Application
$Global:Recycler = $Shell.NameSpace(0xa)

$Global:Recycler.Items() | Remove-Item -Recurse
}

I then needed it to run in an elevated state, so I needed to set some options.

$options = New-ScheduledJobOption -RunElevated

Then I wanted a trigger to run every Monday at 7 AM.

$trigger = New-JobTrigger -DaysOfWeek Monday -At 07:00 -Weekly

I’ve also had to have one trigger every hour

$trigger = New-JobTrigger -Once -At "2017-07-14" -RepetitionInterval (New-TimeSpan -Hour 1) -RepetitionDuration ([TimeSpan]::MaxValue)

And then you wire it all together.

Register-ScheduledJob -Name "Recycle Cleaner" -ScriptBlock $scriptblock -Trigger $trigger -ScheduledJobOption $options