Configuration Files, and Why We Need Them
Ever heard of the method of Loci?
It’s spelt with a “c”, not a “k”.
Anyways, the method of Loci is a memory technique designed by a sixth century poet named Simonides. “Designed” might not be the right term for it, since he had more or less chanced upon it. Simonides was a poet, which means he was an artist who composed verses for anyone willing to pay a fee for it because you can either follow your heart and starve to death or follow along and get paid.
He participated in various events to earn his bread and butter, and one of those events included composing and delivering a poem at the house of a wealthy nobleman who went by the name of Scopas. Unfortunately for Simonides, he apparently said something that didn’t sit well with Scopas who then refused to pay half the fee because being a wealthy noblemen isn’t synonymous with being humble.
There is bit more to the story, but what eventually happened was the roof of the dining hall gave in, crushing everyone except Simonides who at the time was chilling outside with his homies because he was built different, unlike the roof, which wasn’t built properly.
Where Are You Going With This?
What happens when people die?
We bury them.
But what if the people there were crushed so evenly that they were hard to tell apart?
We bury them too…but we need to know who is getting buried and where so that the family can pay their respects. As you can imagine, this was a major issue – or at least it would have been.
Simonides managed to recollect where everyone was situated prior to their untimely height reduction. He discovered that he had great spatial awareness and could easily conjure up mental imageries at will. He used his new found superpower to save the day, and that superpower is what we call mnemonics today.
He realized that this superpower was not just available to him, but to all of us, but they don’t teach you any of this at school now do they?
Again, Where Are You Going With This?
When we develop automations, there are details or parameters our bot(s) have to keep track of.
Automating incident creation in ServiceNow might sound easy, but the bot must navigate to ServiceNow and login with the right credentials before it does any of that ROI generating shenanigans.
But before it even reaches there, it has to ensure that it’s accessing the right instance of the application. If the bot gets deployed onto a development server, it shouldn’t access the production instance under any circumstance.
To ensure that our bot works as expected, the bot needs its own Method of Loci to guide it as it traverses across various applications – a sort of mental map for it to rely on.
The easiest, or probably the rookiest(legit word by the way) way to go about approaching this is to chuck the mental map aside and hardcode everything.
Isn’t That a Bad Idea?
Damn right it is.
RPA solutions are built and tested against mock servers/applications before deploying it into the warzone. If we hardcode everything, then we have to hardcode it once more when it’s ready for production.
Making minor tweaks or enhancements to the process will be incredibly time consuming and error prone because you have to make those changes in the code each time.
A lot of unnecessary complexity can be easily mitigated once we fashion a mental map for the bot to work with.
Fashion Design a Mental Gown for the Big Day
Configuration files contain data that instructs the bot on how to proceed, which actions to take, and how to handle success or failures. Don’t get it twisted – the logic is developed in the bot but the parameters, i.e., data comes from the configuration file.
If the bot must enter logs into database, we can maintain a Boolean parameter in the configuration file.
If the bot should access a demo instance of an application, the URL can be stored as a parameter in the configuration file.
If an the automation fails, we can program it to retry by providing a retry parameter in the configuration file.
I’ve started three sentences using If, can you guess why? Yes, my English is not as good as I wish it was, but that’s not it.
Once we retrieve the parameters, we will use several If Actions within the bot to evaluate the parameters so that the bot picks the right branch of execution.
Here’s another example just for the heck of it.
Say I develop an automation and it runs successfully because of course it would – I’m the one who developed it.
The last action it performs would be to draft a mail to the stakeholders along with some relevant details which could either be a summary report embedded within the mail body or an excel spreadsheet. In this case, the email addresses and email body are stored in the configuration file as parameters, while the logic for drafting emails is encased within the bot.
Don’t confuse the two.
Similarly, based on the environment a bot is deployed onto, the relevant email addresses will be picked up from the configuration file and passed into child bots.
Don’t worry, I will show you how that’s done, but before we go into any of that, lets address a question that some of you might have which is:
What Sort of File Should I Use?
XML is a standard we follow, but it’s entirely up to you. Some are comfortable working with Excel, CSV and sometimes JSON.
I will explain how XML configuration files work in this short tutorial, and maybe create a follow up to address JSON and not Excel or CSV.
Why Not Excel or CSV?
Both are fine, but it’s nowhere near as convenient as XML or JSON.
Let’s head back to our previous example. Say we have three sets of email addresses that correspond to Development, UAT and Production. While we can figure out a way to include these details into a workbook or CSV file, it’s going to lead to unstructured complexity.
As you can see, stuff is all over the place and we have to keep track of every key which is one hell of a headache.
The smarter ones will maintain different excel/CSV files or sheets for each instance so as to reference the same keys across all instances without having to enter the code. Instead of devUrl, the key can remain as Url and the production configuration file/sheet will assign the right values to it.
However, the big brain ones will use XML.
Why XML
There is a reason most software or languages prefer working with XML or JSON, and it’s because we can create trees or hierarchies which is really convenient.
These trees might be complex, but they are structured which helps us out a lot. It’s easier to navigate across something when it has structure as opposed to a large blob of data with numerous keys to keep track of.
Since we are working with XML, the way we will query data from it is by using something called XPaths.
XPaths aren’t Limited to Just Web
XPaths aren’t just used for traversing across the DOM. It works really well with XML as well.
If you are new to the concept of XPaths, then watch this video. If you’ve explored it earlier, then you can review whatever you’ve learned so far by reading this article.
That’s right, I have an article for just about everything.
Ok, maybe not everything.
Let’s look at a skeletal sample of our XML configuration file.
<?xml version="1.0" encoding="utf-8"?>
<root>
<config environment="DEV" controlroom="https:\\tct-dev.automationanywhere.com">
<Email>
<SuccessEmail>
<From_EmailID>fromtct@notgmail.com</From_EmailID>
<To_EmailID>totct@notgmail.com</To_EmailID>
<CC_EmailID>cctct@notgmail.com</CC_EmailID>
<BCC_EmailID>bcctct@notgmail.com</BCC_EmailID>
</SuccessEmail>
</Email>
</config>
</root>
XML is comprised of nodes and each nodes may or may not contain attributes.
Similarly, we will have nodes with the same structure for UAT and Production, and the bot will automatically query data from the right node, but that will require a bit of creativity from our end.
Before we query data from it, we have to validate whether its a fell formed XML, because if it isn’t then the bot should terminate execution.
Once we validate that, we can move onto the next step which is to conditionally query data from it.
Automation Anywhere has a wide list of System Variables that comes in handy, especially in situations like these. If you want to check which Control Room the bot is being deployed from, we can use the $System:AAControlRoom$ variable to determine that.
The same variable can be wrapped up into an XPath to conditionally query data like so:
//config[contains(@controlroom,"$System:AAControlRoom$")]
Since we are interested in fetching the Attribute Value, we have to provide the Attribute that we want to fetch the value from. We won’t have to do this for fetching values between nodes.
Lets try fetching the From_EmailID to demonstrate how that’s done:
//config[@environment="$vEnvironment$"]/Email/SuccessEmail/From_EmailID
The Environment fetched earlier will be used as a reference in all the other XPaths. This way, the bot will fetch the right set of parameters without us having to develop any sort of intricate logic within the bot.
I think you’re starting to see why this is far better than relying on workbooks or static files.
Here is a sample XML file for your reference:
Where Do I Store These Parameters?
If you are retrieving 10-20 parameters from the configuration file, you still have to keep track of them while you pass then into child bots. This too can be simplified by converting them into Key-Value pairs, that way all you have to do is pass in that one variable.
That variable is called a Dictionary.
But then again, you might prefer storing them in individual string variables because you couldn’t be bothered to keep track of all the keys in a dictionary.
Its up to you.
What “Can” I Store in Configuration Files?
Details that usually go into a configuration file are website URLs, filepaths, email ids, log levels etc – basically anything that isn’t sensitive.
Don’t be an Idiot and store credentials in configuration files. We have a Credential Vault for that. Credentials include Usernames, Passwords, API Tokens, URIs etc.
What If I Wanted to Store HTML Content for Mails?
That can be done, but do keep in mind that you can’t use angle brackets since the compiler will assume they belong to the XML. You have to rely on substitutes below to represent an angle bracket in HTML when its embedded in XML:
Left Angle Bracket(<) -> <
Right Angle Bracket (>) -> >
If you want to add breaks in HTML you have to use <br>.
If that HTML is embedded in XML, then this is what you have to use instead:
<Email_Body>
Hi Team,
<br><br>
Bot has finished execution, please find the execution report in the attachments.
<br><br>
Regards
AA Bot: {{USERPROFILE}}
AA Machine: {{SERVER}}
*** This is an automatically generated mail, please do not reply back to it.
</Email_Body>
In Conclusion
Configuration files are a vital tool for Automation. In fact, they aren’t just used in RPA, but are used across the board in various software and programming languages.
How you go about designing your configuration file depends on the architecture as well. If your customer has a single Control Room, then you have to define your configuration file accordingly. Maybe you can check which user profile the bot has logged in with, or fetch the server name using the System: Get Environment Variable Action.
As you can see, configuration files are easy to work with, while also making our work as developers easy.
Let me know what you think in the comments below.