Feels like I’m repeating myself, but Cloud isn’t easy

Only two days ago I wrote about deciding to rewrite the Report Generator to be an Azure Function rather than a container. Today, I changed my mind.

The issue we want to solve, primarily, is that generating a report takes too long. Secondary we want to get rid of the LibreOffice dependency.

Turns out, the component needed to replace LibreOffice is expensive, like really expensive. The cheap ones starting at around $800 per year, sometimes per developer and/or deployment. The most expensive I’ve looked at is $10000 per year, but that’s all inclusive.

The estimated cost of running a small container instance with what we need, including LibreOffice, is less than $100 per year. We will have to handle the scaling, rather than trust Azure Functions to scale for us.

Having the Report Container(s) always on will solve the time issue. The LibreOffice issue remains. I’d prefer not to depend on LibreOffice, but it’s not that bad, and since the performance are comparable, I find it really hard to motivate the extra cost of buying a component.

The flow for the user will now be something like

  1. The user clicks a button to generate a report
  2. The Angular app calls the Api (an Azure Function)
  3. The Function stores the Crawl Session to a database
  4. The Session Id is put on a Queue
  5. A result is returned to the UI so that the user knows the report is being generated

The flow in the container will look like this

  1. Application starts, and subscribes to the Queue
  2. A message is put on the queue and the Subscribe event is triggered
  3. The Session Id is read from the message
  4. The Session data is loaded from the database
  5. The Crawler Application is started with relevant parameters
  6. The Application generates the Report and stores it to a Blob Storage
  7. The Blob Storage sends an event to an Azure Function that updates the Session Status in the Database

If (when) we need to scale this, we just add another container that subscribes to the Queue. This makes the Queue a single point of failure, and I need to further investigate how to handle this. For short outages, just letting the User know that the job wasn’t queued, or that it failed, should be acceptable. Short in this context is that when the User tries again, it works.

About Stefan Bergfeldt

Jag som kallar mig för Ordbajsarn heter egentligen Stefan Bergfeldt. Jag föddes på Falu lasarett i augusti 1978, och är uppväxt i Hedemora. Webbutvecklare, sökmotoroptimerare, entreprenör och gitarrist är andra saker man kan kalla mig, om inte Ordbajsarn passar. Jag driver konsultfirman CRS Webbproduktion, och har specialiserat mig på att ta fram kostnadseffektiva webblösningar till små- och medelstora företag.