r/SysAdminBlogs • • 11h ago

I made free escape rooms for Microsoft admins: fix a broken tenant before the clock runs out

6 Upvotes

After the Learn post a few days ago, I wanted to try something less like studying and more like the actual job.

So I made escape rooms. Each one is a made-up Contoso tenant that's broken in one specific way. You get simulated sign-in logs, audit logs and settings, and a timer. You have to find the cause, fix it without making things worse, and then prove it works.

The bit I like most: shortcuts cost you. Delete the Conditional Access policy that's locking everyone out, or exclude someone from MFA to stop the prompts, and it works, but it logs a security incident against your score. Just like real life, the quick fix is usually the wrong one.

The five rooms so far, easiest first:

  • Nothing Enrols: new laptops join Entra but never enrol in Intune
  • The Expired Secret: the HR sync dies overnight with an AADSTS error
  • Locked Out: a Conditional Access policy locks out everyone, admins included
  • MFA Storm: the finance director approved a 3am MFA prompt, and the attacker is already in
  • Mail Is Down: a domain move leaves MX, SPF and DKIM pointing the wrong way

Each room ends with a debrief: what happened, how you'd fix it for real, and links to the Microsoft Learn docs. It's free, with no sign-up and no ads, and your times stay in your browser.

I'm adding 5 new rooms every week. If you've got a real outage story that would make a good room, tell me and I'll turn it into one (anonymised, obviously).

azureblog.co.uk/play


r/SysAdminBlogs • • 6h ago

Grafana Loki – A Practical Guide to Centralized Log Management

6 Upvotes

Hi everyone,

I've been exploring centralized logging and Grafana Loki, particularly how it can help Linux administrators and DevOps teams troubleshoot issues across multiple servers and applications.

One thing I find interesting about Loki is that it doesn't index the full content of every log line like traditional logging systems. Instead, it indexes labels, which makes its approach to log storage and querying quite different.

In this guide, I've covered:

  • How Grafana Loki works
  • Loki architecture and its key components
  • How labels and log streams are used
  • Troubleshooting when logs aren't appearing in Grafana
  • Investigating slow log queries
  • Understanding high memory usage and label cardinality
  • When Loki makes sense compared with traditional logging solutions

I tried to keep it useful for people who are learning Loki as well as administrators looking to understand how it fits into their monitoring and troubleshooting stack.

I'd appreciate feedback, especially from anyone using Loki in production. How are you managing centralized logging today? Are you using Loki, ELK, or another solution?

Full guide:
https://datadevblog.com/grafana-loki-tutorial-guide/?utm_source=reddit&utm_medium=social&utm_campaign=grafana_loki&utm_content=post

Thanks for sharing your experience!