Improving Organisational Intelligence & Insight with Optimised Data Warehousing and Architecture Processes
Government Case Study: Warehouse Transformation: Securing Buy-in for Data Warehousing and Business Intelligence Transformation
Steve Armstrong
Director, Strategic Planning & Capability Enablement
WorkSafe Victoria
Getting the most from data
For most organisations, particularly public sector agencies or large companies, data supports much of the work that these organisations do. In fact, without good data or an appropriate way to obtain data, many organisations would fail to achieve their aims. This is certainly true for WorkSafe Victoria. Steve Armstrong , the Director of Strategic Planning & Capability Enablement at WorkSafe Victoria, says that “data and analytics underpins WorkSafe Victoria’s purpose.” The agency is the state’s “occupational health and safety regulator and workplace injury insurer.” Its purpose is therefore is to “reduce workplace harm and improve outcomes for injured workers.” Data and information management are used to “predict risk in the workplace and optimise the recovery journey for injured workers.”
To further develop and embed this, in 2019 the agency formed “the Intelligence Management Function to uplift advanced analytics capabilities and respond to business-critical questions.” As part of this uplift, new teams and areas were created throughout the agency. This included a “data engineering team” to largely deal with some of the legacy tools that the agency still uses, “a business intelligence team, a business analytics team, and strategic planning team,” all of which are there to “introduce new tools and better data assets so that we can leverage them” and develop a better functioning data agency.
A new way of working
One of the reasons for the creation of the new function and teams was because “we’ve got a legacy environment that is siloed, manual and a very people-intensive process.” The output of all the new functions will be “an Enterprise Data Hub that will deliver great outcomes” to both the agency and ultimately the Victorian community.
The hub is comprised of three main components:
A data lake
“A single source of truth where data is governed and trusted across all business domains. It is largely unstructured but with linkages to internal and external data sources, and through cloud computing, it can be scaled and thus reduces demand on IT infrastructure.”
A data warehouse
“This is where data is transformed into something meaningful. It provides a common view on the business and on-demand access to reports and dashboards. We want to move away from more spreadsheets. It gives us an intuitive presentation of business performance across key domains, and allows analysts to perform at their best to identify trends and deliver insights.”
Advanced analytics
“This is the final process that delivers predictive intelligence capabilities, and leverages data at scale to predict emerging risks. It provides personalised experiences enriching products and services, and incorporates machine learning to enable better evidence-based decisions.”
All of this can only work because of the new architecture that is built around the hub to enable and empower it to “deliver incremental value.” WorkSafe Victoria has many disparate parts to its business and all of the parts need to work together with “historical data matched to current data and anything new coming in. There’s obvious complexity in trying to make all of that work and conform it into something meaningful, but the future state with allow for that.” The hub is hosted on the cloud allowing for scalability and is being built using the latest technologies and software processes.
Importantly, the hub and all the new functions are “underpinned by data governance.” There is a new data governance council to oversee the ethics and privacy issues, but “as we build out the platform, we must mature our capability in data governance across all tiers of the environment. That’s one of the core components. There is also a delivery methodology to ensure that this project has a long-term future.
Lessons and learnings
The hub is not yet complete and “we are still on this journey,” but the data lake has been established and lessons and patterns are starting to emerge. For instance, it is “really important to get organised with design patterns.” This means having early agreement on the architectural approach and frameworks. It means “getting buy-in from the team upfront” and “striking the right balance between coding, configuration and tooling.” This may mean that a recruitment drive may be necessary to get the appropriate skills into the team, because ultimately, “you need to bring the people along the journey and they need to be well supported and have the right skills to operate at their best.”
It is also important to “establish a prioritisation framework and a delivery methodology.” This means specifically choosing “your first use case wisely, considering the impacts, risk and complexity.” Early wins are important because they create “incremental value.
”
You’re not going to be able to deliver everything at once, as much as your business would like you to. There are going to have to be trade off discussions along the way. It is best to prioritise at multiple levels and distinguish between what must, should and could be delivered. Make sure that what you put at the top of the list really is something that must be delivered.
Steve Armstrong Director, Strategic Planning & Capability Enablement, WorkSafe Victoria
It is also important to realise that as much as small wins are necessary, sometimes there are “implications of running legacy products and new ones side by side. As much as you might want to turn off the old, there will be an interim time where you will have to operate in parallel and this will slow things down.” It is therefore crucially important to “know your critical data elements.
Bad data in a legacy system continues to be bad data in a new environment.” Whilst there might be a desire to fix everything and improve the data health of the entire system, “prioritise those issues that you consider to have the greatest impact on your environment.”
It is therefore necessary to “understand the purpose of each data source.” One way to do that – especially if there is a lot of data and many data sources – is to “establish data owners, custodians and stewards who know the business and the community they are servicing.”
Often the only part of this process that customers or external people see are the dashboards. It is important to make them look good and appealing, but sometimes “sexy dashboards get too much of the credit, though that’s not such a bad thing.” Stakeholders or anyone interacting with the dashboards will mostly only care for what they can see, and many “love being able to touch and feel and see where all of the data is going.” More than likely, there will be a lot of “sophisticated predictive intelligence and amazing data modelling” in the background, but
“ultimately, businesses love dashboards and all they can visualise, so aim to get to a point of visualisation as early as possible.”
That doesn’t necessarily mean “rushing to get our dashboards done,” but it does mean “getting our best analysts in there and getting them to start solving business problems.” The results can then be displayed and presented on dashboards in ways that stakeholders can appreciate. “We want to make it really easy for the business and stakeholders to understand where they need to look and what the business impacts are of any issues.”
