As experts in your respective domains, how do you balance the need for innovation with the stability and reliability of existing systems?

The purpose of innovation is to create (or preserve) value for your organization. The most difficult part of innovation is measuring the impact (as Robert indicates) which takes a lot of discipline. Balancing innovation against existing operations is a function of how disciplined the org is when it comes to this. Many orgs are really looking to "innovation" for a splash of creativity and not really trying to tie it to performance. 

I very much agree with Victoria. Though I have one addition. Really understand the outcome that you are trying to achieve. That way you can establish if the changes you are making are having the desired impact.  Often changes to systems are made without a clear idea of "why" and how one would know that you have succeeded.

If you don't know what you are trying to achieve how do you know if its worth the risk.

I would always suggest taking an iterative / agile approach allowing you to test and learn what your impact is on the outcome you are trying to achieve.

With difficulty is the short answer. 

The longer answer is:

Proper prioritization based on assessment of urgency, importance, value, cost and risk. 

Regular and open communication across teams and stakeholders is essential to track changing needs and wants. 

Real understanding of stakeholders (rather than lip service to 'users') is fundamental to success. As is intelligent and disciplined application of methodologies (Agile without discipline is anarchy and agile without flexibility is insanity!). 

Being a slave to an idea, technology, approach or direction of travel is also dangerous, being prepared to revisit decisions and course correct in the light of new information is fundamental - and finally focus is key: understand what you are trying to achieve and why, don't get seduced by anything that does not contribute to the mission - and be realistic about the mission (if you expect to be first in your industry, but are risk averse and always wait for others to act first you are already set up for failure)