Putting the Business back into Business Intelligence: reduce Data Debt by reconciling IT Department with Finance Professionals

Ironically, there are so many technical tools and ways to communicate, but there are still problems in achieving effective internal communication in companies. This is particularly evident in relationships between Finance and IT. In this topic, the issues will be explored further  The issue with having so many technical tools is that the quantity of communication may increase, which can give the appearance that communication is going well. However, the quantity of communication does not always reflect the quality of communication. IT and Finance concepts are complex for non-specialists to understand, and messages may not be received as intended when working across different teams. It is hard for each department to see through the lens of the other department, and this issue can be further exacerbated if there is a culture of blame in place.

What is Data Debt?

Data debt is like technical debt, only visible in bad data, poor data quality, and underuse of data-oriented technology to help make better decisions. It is the overhang of projects, whether they went well or poorly.

Technical debt is the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. It also applies to data debt, the cost of ignoring the data by choosing the quick or easy solution earlier at a previous point, which now takes longer to resolve.

At some point, however, we need to pay back data debt if we are to succeed in making better decisions.

The issues all add up to make data debt, which is often a hot mess that nobody wants to resolve. Businesses can develop the Bystander stance; it is someone else’s problem.

What can you do about data debt?

Finance and IT need one another so that the whole organization can be successful, but there can be issues in communication and priorities that cause divisions. This can be seen very clearly in business intelligence, where the business and IT need to work together to produce an essential component of the organization’s function: reporting, data, and information flow.

Business Intelligence projects can inherit data debt from previous projects, which means that projects do not start from the starting line. Instead, the projects can start with a deficit before making progress. In the long term, technical debt can exacerbate issues between the Finance and IT departments. Take, for example, the situation where Finance doesn’t believe that their requirements and priorities are adequately understood and that their voice isn’t being ‘heard’ in the IT strategy. From the Finance lens, this situation could be shown where IT won’t respond to support calls quickly enough, for example, or won’t allow the purchase of some new software.

On the other hand, IT can believe that Finance is using uncontrolled data sources, such as Excel or Google Sheets spreadsheets. If something goes wrong with these artifacts, Finance team members involve IT to help fix something they do not own and inherit the technical debt for something they did not create. This means that IT can end up owning ‘cottage industry’ artifacts, and this can create frustration. The ‘cottage industry’ artifacts arise partly due to Finance not getting the data or the software they need from IT, so team members create their own data sources. The cycle of frustration between IT and Finance can continue without resolution, and since neither department owns the pain, there is no actual ownership of resolving the issues. Instead, it gets absorbed into the business processes and becomes part of everyday work. The real cost is never identified since it is a productivity cost and isn’t measured as stringently as direct costs that people can see or hold.

Conflicts between departments can arise from differing priorities, but the overall vision should be the same for each department. As IT pushes into new projects, technical debt can be pushed to the side simply to accumulate. For Finance, this issue can develop into unresolved conflict, ongoing productivity issues, and concerns over data governance and veracity. The reality is that IT and Finance need to work together for the success of the whole organization. There are risks if the departments cannot work together. For example, if the departments cannot work together, then this could put security at risk, for example, due to a failure to communicate successfully. Ultimately, this could land the company in court. Other consequences include an adversarial working environment which can make the company unable to recruit and retain key talent. In turn, this can impact technical debt as people leave without resolving it, and it accumulates. Although it’s hard to quantify, the difficulties in IT and Finance working together will cost the organization money, whether it is calculated in terms of lost productivity or direct costs such as poor software selection. By contrast, if the teams can work together and work better, then both departments can increase their contribution to the success of the whole organization.

Blueprint for paying back data debt

How can the teams move forward from the impasse? We must put the business back into business intelligence and move forward, working more collaboratively.  Fundamentally, this is an issue of how information can be tailored to different audiences who are specialists in their field and need to work together for a specific outcome seen across the organization.

Both Finance and IT departments need to work together to improve understanding and collaboration between each department. Brown bag lunch sessions, where team members present bitesize summaries of relevant concepts to both teams, can provide a good experience of working together for all team members. This may also help to break down stereotypes of the characterization of each department. Further, it can help team members to enlarge their tribe to include people from other teams as well as their team.

Finance and IT leaders need to work together and set an example for the rest of the organization. Personal conflicts between department managers is unacceptable, and the dispute will affect processes and practices within the company and disrupt the workflow.

Set up an informal wiki or blog where team members can add daily. This blog can be used to identify the data and the information each department needs every day. By identifying critical information, the wiki can be used to help understand each department better and to share this crucial information in a straightforward, recorded way. This will help the teams see the impact they can have on each other and identify and prioritize the efforts to maximize benefit.

Creating a community of excellence as a cross-department initiative with representatives from each department working on joint initiatives, with progress reports and clear objectives. This will take some time to set up, and there may be resistance due to previous projects that have failed or not delivered.

Inherit technical and data debt. Every new IT project should inherit some technical debt from previous It work. This could involve data cleansing, improved reporting, or the review of a support contract of cloud vendors.

Requirements can be hard to write and accept. One way to increase teamwork is to accept that prototyping is a valid way of developing and articulating what Finance needs. Further, IT can take its considerations, such as security, governance, and upgrades, on board. When developing, IT will want to prototype; at first, the efforts will seem awry, but constant communication and feedback will be a good investment of time.

Conclusion

It isn’t going to be easy, but it is achievable if the example and direction are set at the C-suite level. Communication is key.

And finally, you can see the end of the data debt.