Data blog — data architecture RSS

Assisted data comparison

Data comparison is an often overlooked problem that many IT professionals have to approach in their daily work. Without even realizing it, they spend a significant portion of their time comparing the data. In many cases the data comparisons are implemented ad-hoc: developers, QA testers or DBAs write SQL queries that return rows with differences between data sets. Often the data to compare do not reside in the same data store - they have to be pulled in to a location where it is possible to execute a query. This takes time. This takes mental effort. This creates opportunity for mistakes. It turns out that there are many data comparison tools, varying with their capabilities and price. These tools can...

Continue reading

Table partitioning in relational databases

I would like to write shortly about table partitioning in relational databases. Table partitioning is basically dividing your data in a table into horizontal chunks, that can be (depending on the DB technology you use) indexed separately and stored on different disks. This allows you to address certain performance issues, if a table is large, and there are many inserts into it, and there is a requirement of providing reports on the data in this table. That table might be for example a transaction registry from your retail network. Partitioning allows separate 'read only' partitions from the active partitions. For example if you insert a lot of transactions to your registry table, they usually have a timestamp associated with them. You...

Continue reading

Graph extensions in SQL Server 2017

We have been talking a bit about graph databases in recent posts. Data Modelling and Monty Python What are the advantages of using a graph database? The rise of the graph databases. Well, it turns out that also SQL Server jumps on the bandwagon and makes first steps towards graph processing.  In the first release, the functionality will be a bit limited, but you can write some workaround code for certain types of graph traversals. The functionality of the first release is explained in the FAQ article on the Technet blog.   There are advantages of the SQL Server solution. SQL Server has become an industry standard and one of the main players on the RDBMS market. The graph functionality...

Continue reading