Model-Based Systems Engineering – Measuring Success

Today’s blog post is a continuation of a series on Systems Engineering for Smart Products. Remember the old Xerox commercial featuring a monk tasked with making 500 copies of a multi-page, handwritten document? Well, fast forward to 2014 and replace the monk with a systems engineer verifying hundreds of requirements against a textual-based description of a product, and you have a typical scene playing out across many engineering enterprises.

In the end, however, the systems engineer will not be rescued by a photocopier, but rather by Model-Based System Engineering (MBSE) methods and tools.

roi model based systems engineeringAs with any new concept, MBSE must prove its worth by demonstrating significant ROI. At ANSYS, we are seeing more and more projects and pilot programs deploying model-based systems engineering practices. I can tell you that the results I have learned through our customers is remarkable. In some cases, the complexity of the project was so great, that a model-based approach was the only solution. That is an ROI of infinity.

I’m not at liberty to share hard numbers, and even if I did you wouldn’t believe me, or they wouldn’t apply exactly to your situation. Instead, I offer you some guidance on how to best measure ROI for your MBSE initiative and what to look for in an evaluation.

Quantifying Time Savings
Most MBSE projects I’ve seen qualitatively report a TTM savings. If quantitative measures are desired, I’d focus on measuring the calendar time between design gates. This can be directly traced to MBSE, whereas TTM can also dependent on an array of non-engineering activities.

Keep your eye on three areas for significant time savings — requirements verification, embedded software production, and verification and testing activities. Also, be careful not to confuse time spent on system analysis versus time spent on generating system models. Model generation and maintenance will be new areas of investment for MBSE processes, but the system analysis should be occurring whether the process is document or model-based.

Quantifying Productivity
Productivity gains attributed to MBSE are best measured by examining the number of design iterations required to get to the same level of quality versus a document-centric process. This is a measurement which is easily captured in most projects.

Look for significant productivity for software engineers due to the direct code generation. Productivity gains are also typically seen with producing and managing interface control documents, as well as documentation for certification purposes.

Be aware that MBSE may not appear to improve productivity for system engineers. System engineers may feel burdened with building and refining models. Keep in mind that this is the down payment for greater efficiencies in other areas, as well as an investment that can be re-used in future projects.

Quantifying Quality
This one is simple. The impact MBSE has on quality is best measured by number of defects, a metric universally tracked by system engineering processes. Most observe that a system described by models tends to be more explicitly defined and communicated, thereby reducing late stage defects.

As I said in my previous post, don’t embark on MBSE just because “everyone is doing it”. Develop a clear understanding of the end goal — whether it’s reduced time to market, increased productivity, or improved quality — and measure your ROI.

Related blogs:

Deploying Model-Based Systems Engineering within the Enterprise
Systems Engineering for Smart Products

This entry was posted in Embedded Software and tagged , , by Todd McDevitt. Bookmark the permalink.
Todd McDevitt

About Todd McDevitt

Todd McDevitt is a Marketing Director at ANSYS, Inc. and has more than 15 years experience designing and developing commercial engineering software. His areas of expertise include nonlinear finite element analysis, multibody dynamics, embedded control systems, and model-based system engineering. At ANSYS his efforts are directed towards strategic marketing initiatives and facilitating the company’s R&D planning and decision-making. Mr. McDevitt received his Ph.D. in Mechanical Engineering from the University of Michigan and holds a Professional Engineering license in the State of Michigan.