I have heard that, to change my name on my bank account, I cannot use the Marriage Certificate presented to us on the day of our marriage. Is this correct?

Yes this is correct.   The Marriage Certificate presented on your wedding day is merely proof that your marriage took place.  The reverse side of the Marriage Certificate advises of this situation. 

What you will need for change of name purposes is the registered version of your Marriage Certificate from the Registry of Births, Deaths and Marriages (BDM).  BDM, however, charge a small fee which normally is over and above the fee you pay to your Marriage Celebrant.  However, many Marriage Celebrants will be happy to facilitate your application to BDM if you wish, and in this way they would act as a conduit between yourself and BDM by submitting your application (and payment for same) at the same time as they lodge the other marriage documents. 

Additionally, you should also be aware that you will need the BDM registered version of your Marriage Certificate to change your name on your passport and on your driver’s licence.  Professional Registration Boards (e.g. Medical, Nursing) and some employers also will only accept the BDM version for change of name purposes.  Please check with your employer what is the case in your situation.

If none of these situations present (for example you are not changing your name), you are not obligated to obtain the BDM registered Marriage Certificate.  This would then become a matter of choice.