|
|
|
|
Main
Page by Topic |
||
|
C. Statistics |
||
Vendor
Risk for CTRM Systems – Audit and Audit-Assist Checklist
See Also: Vendor
Risk Management
See Also: Vendor
Risk Mitigation
Overview
This page
provides recommended practices for a firm using a Big CTRM system around the
concept of a ‘Vendor Risk Audit’, also known as a ‘Vendor Risk Assessment’, and
provides a must-see checklist to
help with that.
The Checklist
helps you save time by giving you the best starting point for an easy, speedy
and comprehensive review.
The results of
following the Checklist, i.e., the responses that you get, form the contents of
your Vendor Risk Registry and Mitigation Plan.
Background – The
Initial Vendor Review
We are all
familiar with the idea of a trading firm including a Vendor Risk Assessment as
part of a Vendor Selection project for a CTRM system. For example, asking about the financials of
the Vendor to make sure they are well capitalized.
It especially
important for firm when there is a big-budget project or there is a long term
strategic decision at stake. Firms want
to ensure that the significant resources they dedicate to a particular CTRM
software package are not wasted.
Period Vendor
Risk Audit
The risk
profile of a particular Vendor and their software can change over time. So if it has been a few years since your
initial implementation, now is the time for you to review the current situation
with your Vendor.
Industry best practices
call for a Vendor Risk Audit every two years.
Or sooner in the case of a material event, such as if your CTRM Vendor
is acquired.
The idea is to
revisit your initial Vendor Risk Assessment and update it to keep it current as
times and situations change.
Vendor Risk
Registry
The Vendor
Risk Registry document is what you when you complete the Checklist below. It contains your assessment of your risk
level as well the Risk Mitigation strategies you are pursuing.
Vendor Risk Registry
and Mitigation Plan as CYA
The primary
purpose of having a Vendor Risk Registry document and Mitigation Plan is
avoiding unwanted and unwelcome outcomes in the first place.
However, if a
Vendor Risk Event happens, such as the Vendor de-supporting a certain software
package that your firm has invested significate resources in extending and
customizing, having an up-to-date Vendor document, with the information covered
by the Vendor Risk Audit-assist checklist can help a person in the systems-ownership
hot seat.
Vendor Risk
Checklist and Planning: Benefits
Benefits
include:
1) By
understanding with more clarity your firm’s Vendor Risk Profile, your firm can
hopefully avoid issues in the first place, i.e., issued caused by a Vendor Risk
Event or similar situation.
2) If there is
a Vendor Risk Event, having a better understanding of things as well as
following appropriate Vendor Risk Mitigation strategies will help a firm move
forward.
3) Help avoid
‘gotchas’ from a Vendor’s licensing terms.
4) Help avoid
wasting resources by putting too much resources into a product that has limited
Vendor and/or future market support.
Vendor Risk
Checklist
Note: The full
description for the Checklist Items including instructions and comprehensive
and clarifying information follows, just after the Checklist below.
Checklist Section 1) CTRM Vendor Health
Section 1
items are the ones you simply ask the Vendor for Info. A well-run Vendor Firm should have this
information at the ready, meaning it shouldn’t take much time on their end to
get this to you for your records, i.e., your Periodic Vendor Risk
Assessment/Audit.
|
# |
Item |
Instructions/Notes |
|
1.1 |
Financial
Health Check |
Ask for the same
sort of Financials you would ask in an Initial Vendor Selection. |
|
1.2 |
Product Road
Map |
Ask for a
Production Road Map of the planned features. |
|
1.3 |
User
Conferences |
Get the planned
user conferences schedules and note if the trend is stable or negative. |
|
1.4 |
Product
Development Resources |
Ask about
and look for negative changes to either the number of people or the quality |
|
1.5 |
Technical
Support |
Ask about and
look for negative changes to the level of support and the Help Desk |
|
1.6 |
‘Nickle and
Diming’ |
‘Nickle and
Diming’ in the practice of charging for things that used to be or are
expected to be free. |
|
1.7 |
Sales
Resources |
Ask about and
look for Sales Resources jumping ship.
This can be a forward-looking indicator of a future Vendor Risk Event. |
|
1.8 |
Older Tech |
Make an
assessment if the technology used by the software is considered modern and
up-to-day. |
|
1.9 |
‘Spaghetti Code’ |
Ask about
the maintainability of the code base.
Hard-to-maintain software limits the productivity of the development
resources. |
|
1.10 |
Relationship
with Consulting Firms |
Ask your Vendor
about their relationship with Consulting Firms in general as a matter of
philosophy and with specific firms.
Look for negative changes as increasing Vendor Risk. |
Checklist Section 2) Licensing
|
# |
Item |
Instructions/Notes |
|
2.1 |
License Key |
Document the
license key situation with your Vendor, especially with regard to expiration
dates. |
|
2.2 |
Licensing
Type. Perpetual? Term? |
Describe the
Type of Licensing you have with your Vendor. Is it perpetual? Or for a limited term? |
|
2.3 |
Term License
Renewal Terms and ‘Gotchas’ |
For firms
who have a term license, e.g., for certain number of years. Document the details. Including any ‘Gotchas’. |
Checklist Section 3) Firm-specific Risks
and Planning
|
# |
Item |
Instructions/Notes |
|
3.1 |
Create a
Plan |
Create a
documented Plan describing the Vendor-risk mitigation strategies you are
employing |
|
3.2 |
Risky
Software Features |
Identify the
top 3 to 5 parts of the system that are risky to the firm. |
|
3.3 |
Custom code |
Document the
extent of your custom code. This
applies to CTRM systems that have a built-in programming language. |
|
3.4 |
Consequences |
Describe and
document the consequences for your firm if the Vendor goes out of business or
if you otherwise lose support for your CTRM system. a) in a
2-year time Horizon b) in a
5-year time Horizon |
|
3.5 |
Strategic
versus Tactical |
Document for
internal-to-your-firm consensus and planning if the CTRM system is viewed as
‘Strategic’ or ‘Tactical’. |
Vendor Risk
Checklist – Full Description of Each Item in the List
Checklist
Section 1) CTRM Vendor Health
1.1) Vendor
Financials
Ask for the
same sort of Financials you would ask in an Initial Vendor Selection. Check Capitalization and if the Vendor is not
carrying a risky debt load.
1.2) Product Road Map
Ask for a
Production Road Map of the planned features.
With a healthy vendor, they should be able to provide you with a good
plan for the product’s future.
If they don’t have
a good plan, that could mean their product is at risk. For example, if the Vendor acquired multiple
similar products, there is a question if they’ll favor one over the other. You’ll want to avoid the case where the
software your firm is using is the one left to atrophy away.
As a side
benefit, getting the Product Road Map can help your firm feel good about your
annual investment in maintenance payments, i.e., feel that you are getting your
money’s worth.
1.3) User Conferences
Typically
Vendor will have period user conferences, customer roundtable and/or similar
type events to allow for their clients to
a) Share
experiences with other users
b) Interact
with the Software Designers and Developers and
b) Coordinate
and/or vote on preferences for future enhancements
As part of a
Vendor Risk assessment, see if such activities have been and continue to be
stable. An example of a ‘negative event’
would be if the software your firm uses previously got a dedicated user
conference and now gets a shared user conference with other software packages
that the Vendor supports, perhaps from an acquisition.
1.4) Product Development Resources
Ask about and
look for changes to either the number of people or the quality.
a) For
example, if the number of people working on your software packaged dropped from
100 to 20 since the last Vendor Risk Assessment.
b) The
headcount number only tells part of the story.
There is also the quality of the people.
Has the firm experienced a Brian Drain where the original designers and
builders of the software have left, leaving a new set of people who don’t
understand things so well?
c) What you
are interested in here is how many people have a Working Knowledge of each
particular area of the system. People
with a Working Knowledge already understand how it is designed and coded and
can get right to work on bug fixes and enhancements. New people who don’t have such a good
understanding will need to spend some time upfront researching things, reducing
their productivity and effectiveness.
1.5) Technical Support
Ask about and
look for negative changes to the level of support and the Help Desk.
For example,
in previous years the front line person that you would call and talk with would
have some product knowledge and be able to help directly. And maybe that deteriorates such that when
you call in you just get a call center.
Or perhaps the Vendor takes away the ability for firms to call in to a
Help line.
1.6) ‘Nickle and Diming’
‘Nickle and
Diming’ in the practice of charging for things that used to be or are expected
to be free.
Here is an
example of that practice.
Previously:
Your firm used to be able to contact the Vendor’s Technical Support Help Desk
and the front line support person could bring in the original developer onto
the call to help answer a question or resolve an issue.
Currently: The
Vendor stops that friendly practice and instead makes your firm pay an hourly
rate for everything, including basic questions about functionality.
1.7 Sales Resources
Ask about and
look for Sales Resources jumping ship.
This can be a forward-looking indicator of a future Vendor Risk Event.
1.8 Older Tech
Make an
assessment if the technology used by the software is considered modern and
up-to-day.
1.9 ‘Spaghetti Code’
Ask about the
maintainability of the code base.
Hard-to-maintain software limits the productivity of the development
resources.
1.10 Relationship with Consulting Firms
Ask your
Vendor about their relationship with Consulting Firms in general as a matter of
philosophy and with specific firms. Look
for negative changes as increasing Vendor Risk.
In fact, there
is no bigger single indicator of a CTRM Vendor where ‘the walls are closing in’
than having the situation where they shut out the Consulting firms, i.e.,
System Integrators and System Customizers, so that the CTRM firm can try to
make a bit more cash in the current fiscal year.
In other for a
CTRM system to be successful in the long term, you need an ecosystem of
consultants around it, trained up in the system and with a corresponding
knowledge of a trading firm’s business.
This is
especially true for Commodities due to the strong localizations needed by
market. For example, how you trade
Mexican Power can be quite different than the European markets for Power, e.g.
Nord Pool. Or variations by commodity,
e.g., special considerations for Softs, e.g., Cocoa, versus Natural gas.
No CTRM firm
is going to have subject matter expertise in all of the commodities and of the
localized way that they trade. That is
one of the key areas where consulting firms fill the gaps.
If you think
of the total costs for a Big CTRM implementation as ‘a Pie’ and each slice
represents a part, including
a) the
licensing fees
b) the
consulting fees around configuring and customizing the software
c) other items
such as building interfaces for downstream consumers, trade feeds and prices
feeds
d) hardware
costs, such as for hosting, e.g. managed service for hosting in the Cloud.
Then having a
vendor switch to a strategy of trying to get ‘the whole pie’ for themselves it
one of the biggest ‘red flags’ in terms of Vendor Risk.
When a firm
shuts out consultants, what happens is that the consultants in the industry,
valued because of their market and locality-based knowledge of commodities,
shift to coalesce around competing systems, and there are many in this
market.
That leaves
the short-sighted Vendor, yes, getting more of ‘the pie’, but, for them, it is
not good news, since their ‘Pie’ get smaller and smaller over time, relative to
what it would be if ‘software companies focus on the software and consulting
companies focus on the hardware’.
Checklist
Section 2) Licensing
2.1) License Key
Does your CTRM
system require a license key from the Vendor?
Does it have an expiration date?
Such that the system will stop working after that date? What would happen if the Vendor is not around
to provide an updated key?
2.2) Licensing Type – Perpetual or Term
Software
licensing will come in two flavors:
A) Perpetual:
One original licensing Payment. And
then, typically, annual maintenance payments of 15% to 30% of the original
License Cost. E.g., if the original
license is $50k per person, maintenance at 20% would be $10k/year.
With a
Perpetual License, firms can continue to use the CTRM software forever, even if
they stop paying the annual (or periodic) maintenance. However, if they do stop paying the periodic
maintenance fees, they would typically lose access to the Vendor’s Technical
Support Desk and lose access to new versions of the Software.
B) Term. With a Term license, firms immediately lose
the right to use the software at the end of the Term. That could also mean losing access to the
software, e.g., if the software is hosted on the Cloud by the Vendor, or if a
License Key expires, limiting access to various features of the software.
2.3) Term License Renewal Terms and
‘Gotchas’
Carefully and
attentively document any of the ‘Gotcha’ items from the Licensing Agreement
with your CTRM Vendor.
This is often
called ‘the fine print’, referring to provisions in the contract which are
buried deep in the document and/or otherwise are not highly visible.
This is
especially when working with a Big CTRM vendor, where the relative power of
each party often favors the Big CTRM Vendor.
As an example,
if you have a multi-year Term licensing agreement with the Vendor, when would
you need to give notice that you don’t plan to renew.
As a very scary
scenario for any firm, under some onerous terms, it may be possible for a firm
to switch off a Bit CTRM vendor while in the middle of a multi-year licensing
term and, not only have to pay for several years additional, even after they
stopped using the system, the firm may be committed, unawares, for another
term, if they don’t take the extra step of submitting the legal notice to the
Vendor that they do not want to auto-renew.
Checklist
Section 3) Firm-specific Risks and Planning
3.1) Create a
Plan
Create a
documented Plan describing the Vendor-risk mitigation strategies you are
employing
3.2) Risky
Software Features
Identify the
top 3 to 5 parts of the system that are risky to the firm.
Common parts
include:
a) Trade
interfaces from an exchange
b)
Confirmation interfaces, e.g., to ICE eConfirm
c) Support for
third-party software like Windows, Citrix, Oracle or MS SQL.
d) Support for
any Regulatory feeds, e.g., CFTC big trade, FERC reporting, etc.
3.3) Custom code
Document the
extent of your custom code. This applies
to CTRM systems that have a built-in programming language.
Some firms may
have 100s of thousands of lines of custom code, written using a Vendor’s
proprietary APIs.
Of all of the
items most at risk, for many firms, this is by far the biggest risk to their
Vendor.
If they feel
uncertain about the future of their Vendor, especially a Big CTRM vendor, firms
should consider stop coding new business logic in the Vendor’s proprietary code
and instead put that same business logic, e.g., for a report or extract, into
one of the many other options available, such as Beacon.
3.4)
Consequences
Describe and document the
consequences for your firm if the Vendor goes out of business or if you
otherwise lose support for your CTRM system.
a) in a 2-year
time Horizon
b) in a 5-year
time Horizon
3.5) Strategic
versus Tactical
Document for
internal-to-your-firm consensus and planning if the CTRM system is viewed as
‘Strategic’ or ‘Tactical’. You might be
surprised how powerful it is it to simply do this labeling.
For a system
considered ‘Tactical’, it is understood that some work may be throwaway work
until a better, long term, strategic solution is found. And that is OK. So long as it is understood and it is a
choice.
For a system
considered ‘Strategic’, a trading firm will view that system as
a) Something
they’ll be committing significant resources to, in terms of budget for
enhancements and upgrades and also in terms of hiring up resources skilled in
that specific system, which are potentially extensive.
b) A system
that they believe will give them a competitive advantage versus other firms
that are taking a more middle-of-the-road type approach to their technology.
For a Tactical system
a) If you have
some medium-sized project, e.g., for 3 months, perhaps you’ll get consultants
to do the work, e.g., if your employees don’t have the time or the skill
set.
b) You may
want to defer expensive, high-risk, all-or-nothing upgrades. E.g., for a strategic system, you might want
to upgrade every 3 years. For a tactical
system, you might be forced to upgrade every 5 years. That is one upgrade cycle, and all of the
costs and risk associated with it, saved, on average every 10 years.
Introduction to
CTRM
Click on this
link for a great introduction to CTRM software: Introduction to CTRM Software