For today's blog posting I thought I'd do something different. I'm going to tell a story about a real project that I worked on to help illustrate a point. Some of my more recent postings have focused on the various ways to deliver software, and some suggestions on how best to run software projects. More specifically, I've advocated an approach that focuses on quality of the software over when it will be delivered.
If you deliver a great piece of software it can potentially be a game changer. Software can make your company more efficient, or give it a competitive advantage. In some cases it can even impact the lives of millions of people, or potentially make you rich. What's important is getting the software right. Of course you, your company, or your client don't have unlimited funds. There does need to be a balance. But what's ultimately going to determine the success of the software is how well it does it's job. If it doesn't do it's job well, the users certainly aren't going to care that it was delivered "on time".
Today we have great agile principles and frameworks that help us deliver software quickly, react to user's feedback, and continually deliver updates based on that feedback loop. These are newer concepts; we didn't always build software this way. If you're unfamiliar with some of these concepts, I encourage you to read up on agile, MVPs, lean, and feedback loops.
Onward to the story. Roughly twelve years ago I worked for a credit union as a software developer. The primary focus of my job was to help maintain a 3rd party banking system. One day my boss came to me with an idea. We had a financial analyst that was doing a lot of ad-hoc analysis, based on the needs of C-level executives and upper management. He had a system of extracting the data from the core banking system into Microsoft Excel. From there he would apply formulas in Excel to do various bits of number crunching. While he was able to get the information that he need, it was time consuming. Also, he didn't have a programming background, so it took him some time to get the various answers that he was looking for. My boss's idea was that with my programming background I could help him create the ad-hoc formulas quicker.
At the time I had recently completed reading a great 1,000 page book on relational databases. It covered concepts such as SQL, indexes, locking, and other relevant topics. One of the smaller chapters near the end of the book happened to touch on some of the basics of data warehousing. This information was still fresh in my mind. After speaking with the financial analyst to understand what was being asked of him, it became clear to me what was needed was a data warehouse. Mind you I didn't have any experience building data warehouses, but I knew enough that these were the exact type of problems they were designed to solve.
Later I followed-up with my boss with the suggestion of building one. Initially he seemed a bit skeptical, although he didn't dismiss the idea totally. Up until that point I had produced a lot of work that he was happy with, often exceeding his expectations, which gave me a bit of clout. Over the course of the next few weeks, I continued to follow-up with him on the idea. I think it was a combination of my enthusiasm, persistence, and his trust in my abilities that he eventually decided to green light the idea.
At first he suggested that we move forward with a simple prototype. We had access to data related to ATM usage. We kept things simple, and I developed several interactive reports that allowed that data to be sliced and diced in various ways, according his specifications. Initially it wasn't even a data warehouse, but rather a series of reports based on transactional data. He was happy with these initial reports, and decided that there was enough promise there that we should move forward with the full-scale project.
Keep in mind I had no experience with data warehousing at this point. Additionally the credit union's database was based on the Pic OS and Pic database. The Pic database is an older multi-value database system that predates relational databases. I only had experience with relational databases, and there aren't more than a handful of people that know Pic. Needless to say there were lots of challenges ahead of us and a lot unknowns. My boss decided that the potential value of the project was worth the risk. In a lot of ways it was a bold decision of his to move forward. He also decided since this was all new territory to us, that we weren't going to have a deadline in mind. We were simply going to work on it until it was done. That was yet another bold decision on his part, and also a very smart one.
I was a full time employee, so they were going to pay me regardless if the project went forward or not. He decided that the potential value of this project was greater then what else I might potentially work on. I'm sure if things went poorly, he would have pulled the plug at some point too. But in reality the risk wasn't that high. It was simply a question of what I'd be focusing most of my time on.
He did bring in a Pic consultant that provided training on the Pic system for 3 days, which I'm sure wasn't cheap but in the big scheme of things wasn't that big of a deal. Over the next few months, I developed a fully functional data warehouse, complete with advanced front end reports. I worked hand in hand with the financial analyst to help ensure that all of the calculations were correct, and he verified the output to the cent.
I don't remember exactly how long it took, but about 6 months later we had a fully functional data warehouse complete with nightly ETL, interactive reports, and web dashboard. Not to toot my own horn, but what we created was really great. I remember when we demoed the system to the execs and they were thoroughly impressed. That was one of my proudest moments of my professional career.
The reason I tell this story, is that in many ways it was unconventional. It didn't come from a request from up above. There wasn't a deadline; we simply worked on it until it was ready. There was no formal project plans, work break down structures, or for that matter much structure at all. My boss determined what had business value, I learned the technology and developed the system, and the financial analyst shared his expert knowledge and acted as QA. We were being agile before it was a thing.
It was a huge success any way that you could measure it. Incidentally at the start of the project, we made no attempt to measure it at all. My boss simply knew that if we were successful, we'd providing a tremendous value. Whether the project took 3 months, 6 months, or even a year didn't matter. In his mind there wasn't any sense in coming up with estimates, and he was right. We occasionally talked about how things were going, but that was about it.
Now I'm not saying that all projects should be run like this one. However, it does show that this type of model can work and be successful. My boss focused on the potential value, and in the end when the vision was realized the value was delivered. That's all that really mattered, and the time we took to work on the project wasn't in any way a burden to the company. While this type approach may not work in all situations, it can work in a lot more situations than you may realize. At the very least we can all learn from some of the ideas and conventions used, to see alternative ways we can build software.
If you have any feedback, I'd love to hear it. Until next time.
If you deliver a great piece of software it can potentially be a game changer. Software can make your company more efficient, or give it a competitive advantage. In some cases it can even impact the lives of millions of people, or potentially make you rich. What's important is getting the software right. Of course you, your company, or your client don't have unlimited funds. There does need to be a balance. But what's ultimately going to determine the success of the software is how well it does it's job. If it doesn't do it's job well, the users certainly aren't going to care that it was delivered "on time".
Today we have great agile principles and frameworks that help us deliver software quickly, react to user's feedback, and continually deliver updates based on that feedback loop. These are newer concepts; we didn't always build software this way. If you're unfamiliar with some of these concepts, I encourage you to read up on agile, MVPs, lean, and feedback loops.
Onward to the story. Roughly twelve years ago I worked for a credit union as a software developer. The primary focus of my job was to help maintain a 3rd party banking system. One day my boss came to me with an idea. We had a financial analyst that was doing a lot of ad-hoc analysis, based on the needs of C-level executives and upper management. He had a system of extracting the data from the core banking system into Microsoft Excel. From there he would apply formulas in Excel to do various bits of number crunching. While he was able to get the information that he need, it was time consuming. Also, he didn't have a programming background, so it took him some time to get the various answers that he was looking for. My boss's idea was that with my programming background I could help him create the ad-hoc formulas quicker.
At the time I had recently completed reading a great 1,000 page book on relational databases. It covered concepts such as SQL, indexes, locking, and other relevant topics. One of the smaller chapters near the end of the book happened to touch on some of the basics of data warehousing. This information was still fresh in my mind. After speaking with the financial analyst to understand what was being asked of him, it became clear to me what was needed was a data warehouse. Mind you I didn't have any experience building data warehouses, but I knew enough that these were the exact type of problems they were designed to solve.
Later I followed-up with my boss with the suggestion of building one. Initially he seemed a bit skeptical, although he didn't dismiss the idea totally. Up until that point I had produced a lot of work that he was happy with, often exceeding his expectations, which gave me a bit of clout. Over the course of the next few weeks, I continued to follow-up with him on the idea. I think it was a combination of my enthusiasm, persistence, and his trust in my abilities that he eventually decided to green light the idea.
At first he suggested that we move forward with a simple prototype. We had access to data related to ATM usage. We kept things simple, and I developed several interactive reports that allowed that data to be sliced and diced in various ways, according his specifications. Initially it wasn't even a data warehouse, but rather a series of reports based on transactional data. He was happy with these initial reports, and decided that there was enough promise there that we should move forward with the full-scale project.
Keep in mind I had no experience with data warehousing at this point. Additionally the credit union's database was based on the Pic OS and Pic database. The Pic database is an older multi-value database system that predates relational databases. I only had experience with relational databases, and there aren't more than a handful of people that know Pic. Needless to say there were lots of challenges ahead of us and a lot unknowns. My boss decided that the potential value of the project was worth the risk. In a lot of ways it was a bold decision of his to move forward. He also decided since this was all new territory to us, that we weren't going to have a deadline in mind. We were simply going to work on it until it was done. That was yet another bold decision on his part, and also a very smart one.
I was a full time employee, so they were going to pay me regardless if the project went forward or not. He decided that the potential value of this project was greater then what else I might potentially work on. I'm sure if things went poorly, he would have pulled the plug at some point too. But in reality the risk wasn't that high. It was simply a question of what I'd be focusing most of my time on.
He did bring in a Pic consultant that provided training on the Pic system for 3 days, which I'm sure wasn't cheap but in the big scheme of things wasn't that big of a deal. Over the next few months, I developed a fully functional data warehouse, complete with advanced front end reports. I worked hand in hand with the financial analyst to help ensure that all of the calculations were correct, and he verified the output to the cent.
I don't remember exactly how long it took, but about 6 months later we had a fully functional data warehouse complete with nightly ETL, interactive reports, and web dashboard. Not to toot my own horn, but what we created was really great. I remember when we demoed the system to the execs and they were thoroughly impressed. That was one of my proudest moments of my professional career.
The reason I tell this story, is that in many ways it was unconventional. It didn't come from a request from up above. There wasn't a deadline; we simply worked on it until it was ready. There was no formal project plans, work break down structures, or for that matter much structure at all. My boss determined what had business value, I learned the technology and developed the system, and the financial analyst shared his expert knowledge and acted as QA. We were being agile before it was a thing.
It was a huge success any way that you could measure it. Incidentally at the start of the project, we made no attempt to measure it at all. My boss simply knew that if we were successful, we'd providing a tremendous value. Whether the project took 3 months, 6 months, or even a year didn't matter. In his mind there wasn't any sense in coming up with estimates, and he was right. We occasionally talked about how things were going, but that was about it.
Now I'm not saying that all projects should be run like this one. However, it does show that this type of model can work and be successful. My boss focused on the potential value, and in the end when the vision was realized the value was delivered. That's all that really mattered, and the time we took to work on the project wasn't in any way a burden to the company. While this type approach may not work in all situations, it can work in a lot more situations than you may realize. At the very least we can all learn from some of the ideas and conventions used, to see alternative ways we can build software.
If you have any feedback, I'd love to hear it. Until next time.
Comments