Everyone knows where you live

Update 16/09/2023: The issue with the ignored captcha value is now reportedly fixed.

Remember the outrage when C-Planet were found to have been processing a nationwide database of voters containing names, addresses and ID numbers, while tagging us with our political party of preference?  

It seems that the problem is way more widespread, and in summary, everyone can easily get to know where you live.

If I were reading this I may say "ok what's the big deal?". But really would you be comfortable if anyone, given your name and surname knows where your full address? Some scenarios:

  • You meet a creep in a cafe or bar? They know where you live.
  • Someone reads your badge when you are at work? They can stalk you back home. 
  • Separating from a violent spouse and settled in a new apartment? They know where you are.  
  • Someone's given a hard time by an authority figure? They know where to send their thank you note.
  • Your name and surname are overheard when on your way to Gozo or at the airport? Someone can ransack your home back in Malta.
  • Gaming 
  • Identity Theft? Running a targeted scam? The list goes on. 

Some Context

Last Saturday, I went to a store I had never been to in order to buy some pet supplies. At checkout I was asked whether I wanted to join the customer loyalty program. "Why not?" I said to myself. I was already ready to spell out my name, but the girl at checkout told me that just the ID card number is enough. She typed in the ID card, and voila, all of my details including my address automatically populated the fields on her point of sale software. Now, where does this information come from, I wonder? Where would such a country database, indexed by ID number, come from?

The Electoral Register

I am 100% sure the data came from the Electoral Register. There is simply no other database that the employee would have used in such a confident manner, knowing with certainty that it would work with whoever shows up in front of her. 

Note that I say "came from", and not that she used the electoral register while registering me. The online Electoral Register allows everyone to search and view voters on a particular street via the 'Search Register' option. But it does not provide any functionality to search anyone by name, surname or ID number. This is actually in line with the General Elections Act which states that the electoral register must be non-searchable, i.e. not capable of being searched on the basis of name, surname or Identity Card number. 

Therefore the shop clearly has a separate database derived from the electoral register containing my (and your) information. 

Let's explore how this could have been done. For the purpose of this article let's shift our mindset towards creating our own 'Netsec Petshop' and think about how we can obtain all of Malta's citizens data electronically, directly from the source. 

Method 1: The painful way

One can just search the Electoral Register locality by locality, street by street, copy paste the voter information, clean it up and repeat. The only issue is there are thousands of streets. It will take you a while and it will be incredibly laborious to copy paste and clean all the information. We need to find another method. 

Method 2: Buy it 

The Electoral Commission website states that "the registers shall be made available for sale as hard copies from the Department of Information."

Hard copies are not what we are after. This is arguably even worse than Method 1.  We need the customer loyalty program for our pet shop electronically remember?

Method 3: Get all the data electronically and build your own database

We know that:

  1. C-Planet got in hot water after it was processing a similar dataset, and an electronic database was found in an open directory listing on their website. I.e. such databases definitely exist.
  2. As I have personally witnessed, this data seems to be strewn about everywhere including petshops!
  3. The Electoral Commission website has this data. We however need not be constrained by searching by locality. Thus we need all the records in order to be able to search by ID Number, Name, and whatever field is available. That would make our powerful Petshop CRM system and it's what the remaining part of this article will focus on.

If one obtains all voter records, they are no longer constrained by the non-searchable aspect of the electoral register. You can search with whatever field you like.

Building Our Nationwide Customer Loyalty Database

The Structure

Browsing the electoral register website on https://electoral.gov.mt/Register/Enquiry we are shown a form similar to the following allowing us to list all people eligible to vote, living in a particular street.

The database appears to be structured in a simple and clear format. We have the registry type (E.g. General Elections register), containing a number Localities, which in turn contain a number of Streets, and finally each Street has a number of citizens associated with it. 

The Captcha 

 


If we navigate through enough pages, you might encounter the 'I am not a Robot' Captcha. Therefore, if we script a program to retrieve all the information, it will likely be blocked. Or will it?

Let's see how far we can go. 

PREAMBLE FOR NON-TECHIES

Your web browser such as Chrome works by doing a number of requests to a server, getting the replies, and showing them on screen. Even to render this page, several requests are made, one to get the text content, and others to get the images and fonts. Special instructions tells the browser where to place each image where and so on. 

The commands that you shall see here are not some leet hacking tools out of Mr. Robot. They are merely replicating what the web browser sends to the server (in this case the Electoral Commission servers). We shall observe what the browser sends and what it receives. The curl tool which is listed in the examples below is preinstalled on MacOS, Linux, as well as Windows 10 command terminals. This just replicates what your Chrome browser does under the hood. As the examples interact directly with the web server, the information will not be rendered nicely in a web browser and shall look raw, just what we need to put into an Excel sheet or another offline database.  

1.  GETTING ALL LOCALITIES

Getting the Localities is just a matter of doing the following query. 

This simple curl command, executable from Linux or Mac OS, gives us back all the localities in a nice JSON format. 

curl -X POST -F "regtype=1" https://electoral.gov.mt/Orchard.ElectoralRegister/Register/GetLocalitiesList

Sample result extract:

{"councilSectCode":"0003","enCouncilSectName":"Birkirkara","councilCode":null,"divisionCode":"08","sortPriorityCode":"2"},
{"councilSectCode":"0004","enCouncilSectName":"Birzebbuga","councilCode":null,"divisionCode":"05","sortPriorityCode":"1"}

For someone with the intent of building their own voter database (which is pretty much all 16 year old + population), it's straightforward to integrate the commands you see here into a Python, C# application, or any other programming language and save the results in a structured database.

The 'councilSectCode' in the results returned is the primary key which shall be used next to get the street names. 

2. GETTING STREETS BY LOCALITY

To obtain all streets in Birkirkara, you need to perform another simple HTTP query. For Birkirkara, we use the 0003 locality code obtained from the previous result.

curl -X POST -F "regtype=1" -F "localityCode=0003" https://electoral.gov.mt/Orchard.ElectoralRegister/Register/GetStreetsList

This results in many results also in JSON format, small excerpt as follows:  

...
{"streetCode":"00032082","streetName":"Triq il- Wied ","councilSectCode":"0003"},
{"streetCode":"00034933","streetName":"Triq il- Wied ta\u0027 l-Imsida ","councilSectCode":"0003"},
{"streetCode":"00032095","streetName":"Triq il- Wied, Santwarju Santa Tereza ","councilSectCode":"0003"}
...

 

3. GETTING ALL PEOPLE LIVING IN A STREET

The next request requires the locality code, which we have already established as 0003 for Birkirkara, and the streetCode, which we shall try out for Valley Road, which from the previous result we determined it to be 00032082. 

Earlier on we mentioned that the "I am not a Robot" captcha can thwart our efforts here. Can we really do this request? This program is a robot after all. Let's give it a try:

curl -X POST -H "Referer: https://electoral.gov.mt/Register/Enquiry" -F "registerType=1" -F "localityCode=0003" -F "streetCode=00032082" https://electoral.gov.mt/Orchard.ElectoralRegister/Register/SearchRegister?page=1

And voila'. We have a list of people living in Valley Road, Birkirkara (Triq il-Wied). 

Note: I have redacted the information to make it non-personally identifiable while driving the point across.

 

No server-side checks are being done to check that this request indeed does not come from a robot. The request above includes nothing that indicates that the browser is coming from a session created by a human. Non techies: an analogy of no server side checks would be a casino that simply does not care, and let's everyone in, irrespective of their age. Even though this request is being done by a robot (i.e. programmatic and not via a human session), it just passes through and the server gives us the results. No geo-location checks are being done either. The same requests will run fine by someone building such a database in St. Kitts and Nevis. 

Note that when browsing the application in a normal browser session, the browser actually does send the g-recaptcha-response. However the server completely ignores it. In fact in the example above we actually omitted sending the g-recaptcha-response entirely. The server does not care.

Pagination

Let's get back to the result the server gave us. It gives us the Name, Surname, House name, and the ID Card number as sampled in the screenshot above. The results returned are in <td> tags which are dead easy to extract. However for streets having several residents, the results are paginated. This is also not a problem to process and extract all pages programmatically. 

As part of the response, the server gives back the number of pages that are associated with this street. You just locate the linkTo property, and loop through it changing the URL query parameter SearchRegister?page=N and you'll then get the entirely of all the people in the street.

Repeat this for all streets, you got a whole town. Repeat this for all towns, you got the entire electoral register, in a searchable electronic format. Note, apart from the missing robot verification, the server does not appear to have any rate limiting measures in place. This would allow anyone getting such a dump to follow through without any hiccups. 

Putting It All Together

  1. Get all localities. Loop through all localities. 
  2. For each locality get all streets.
  3. For each street, get the first page containing the residents' names, addresses and ID number. Also check the pagination linkTo property, and if there are multiple pages just use it to loop through all pages.  

And now you are just steps away from having the shiny nationwide database that you can use for your shop's customer loyalty program (or any other purpose). Put this in an Excel sheet, SQL database or whatever your favourite format is. You can now search anyone by name and surname, ID number, or address.

Note: the act of processing this data is likely illegal (see disclaimer). Same goes for the action of dumping the entire voter register database. The purpose of this article is to show the current state of affairs with regards to your personal data.

Key Takeaways

1. A lot of fuss was done over C-Planet processing everyone's data. As seen above, this data is being handed over on a plate, and if a pet shop has this, I believe that all of our data is in offline datasets pretty much everywhere. The cat is out of the bag. 

2. The "I am not a Robot" security feature is implemented incorrectly and leads to anyone from anywhere in the world being able to dump the whole database of Maltese voters with their ID number, full name and address. The security check is being done client side (browser) only. Seems like someone forgot to check the g_recaptcha_response value server side. You don't even need to specify the recaptcha value, as seen in the examples above, the server does not care about it. This is a serious gaffe by the developers, who intending to block this kind of web scraping but instead delivered a half baked ineffective feature. 

3. This last point I call the 'security' disconnect. I place an equal blame on any security personnel who checked the application. The only security feature deterring bad guys from dumping the entire electoral register had to be tested properly. This is the result of security 'professionals' focusing on the techy vulnerablilities as listed in the OWASP top 10 vulnerabilities, while missing the things under their nose, not even understanding what the application's purpose is, and what data it is processing. If your security professionals have no concept of understanding the application and doing basic threat modelling than you need to seriously re-assess the value being gained from your information security function.

The C-Planet leak was just the tip of the iceberg. After all, as C-Planet themselves stated, the data was dismissed as old, which begs the question, how many other new versions do they have? Where are these located? How many other entities are illegally processing poorly protected databases? How many scammers have such a copy with all of our information? How long will it take until fake tech support agents will correlate this information with phone directories and start personalising the experience for a more convincing scam targeting our elderly population? Apart from the source itself, how many more C-Planet style badly protected databases are out there?

Hope you found this interesting. A little bit of self promotion; if you are concerned about the security of your application and want things properly tested for threats that actually matter, I do penetration tests. Fill the form in the footer of the page. 

Disclaimer

The information provided in this article is for educational and to demonstrate technical concepts. I emphasize that I have not personally carried out or advocated for the execution of any actions as described in the content. I do not endorse, encourage, or condone any illegal activities, including unauthorized access to databases or violation of data protection laws. The content presented here is based on my personal opinions and observations, and should not be considered professional advice or guidance. Any references to specific websites, tools, or techniques are intended solely for illustrative purposes and not as a call to action. It is important to respect the laws and regulations governing data privacy, security, and usage in your jurisdiction. Readers are advised to seek legal counsel and guidance from qualified professionals before attempting to access or use any information or data described in this article. The author and the platform hosting this article disclaim any liability for actions taken by readers based on the information provided. In no event shall the author or the platform be held liable for any consequences, legal or otherwise, arising from the interpretation, use, or misuse of the information presented in this article. Always ensure that your actions are in compliance with applicable laws and regulations.