Canadian Nuclear Safety Commission
Audit of Contracting and Procurement
Table of Contents
Finding 1: Monitoring and Reporting
Finding 2: Performance Measurement
Finding 3: Contract Review Committee and Post-Contract Evaluation
Management Response to Audit Recommendations
Executive Summary
Background
The Audit of Contracting and Procurement was included in the approved CNSC Risk-Based Audit Plan: 2008-2009 to 2010-2011. The CNSC is subject to the Government of Canada’s legislative and policy framework for contracting including the Financial Administration Act, the Government Contracts Regulations, the Contracting Policy, and the Guidelines on the Proactive Disclosure of Contracts. Effective procurement is a key indicator of good stewardship in the Government’s Management Accountability Framework.
Contracting and procurement activities in the CNSC are managed by the Contract Management Services (CMS) unit of the Finance and Administration Directorate. In the calendar year 2008, CNSC awarded 1564 contracts of various types such as competitive, non-competitive, local purchase orders, standing offers and temporary help with a total value of $22,710,850.
Focus of the Audit
The objectives of the audit were:
- to provide assurance to the President and Senior Management that CNSC contracting and procurement function is operating efficiently and economically with effective controls; and
- to determine if CNSC contracting and procurement practices are compliant with CNSC and federal government policies, statutes and regulations.
The scope of the audit included examinations of:
- controls established by the CNSC for contracting and procurement,
- selected procurement and contracting transactions with external suppliers focusing on higher risk transactions for the period of April 1, 2008 to December 31, 2008, and
- contracting systems, processes and files related to the above files for compliance to federal government and CNSC statutes, regulations and policies.
Conclusion
Management controls have been effective in ensuring compliance with legislative requirements for contracting activities. The audit found no examples of non-compliance by the CNSC with the legislative and policy framework of the Government of Canada for contracting and procurement activities.
The CNSC has in place some key elements of an effective management control framework for contracting and procurement activities:
- CNSC policies and procedures are generally consistent with Government legislation and policy requirements;
- roles and responsibilities for CMS staff and for CNSC managers are clearly defined;
- sufficient information on contracting processes and procedures is readily available to employees;
- internal communications in the CMS unit is effective, and;
- training for CMS staff is sufficient.
The audit identified opportunities for improvement in the management and oversight of contracting and procurement activities:
- monitoring and reporting of contracting activity should be improved to ensure transparency and accountability;
- a performance measurement framework should be developed and implemented to demonstrate due regard for efficiency, effectiveness and economy of contracting activities;
- a contract review committee should be implemented to ensure effective oversight and challenge of contracting activities;
- a post-contract evaluation process should be implemented to ensure best value in contracts;
- a formal, documented risk management framework, such as the proposed Risk-Based Approach for Procurement and Contracting, should be implemented to ensure that risks associated with high risk contracting activities are identified, assessed, mitigated, monitored and reported;
- training on procurement planning, requirements definition and contract management should be required for all CNSC managers.
The audit team is unable to conclude whether CNSC contracting and procurement activities are operating efficiently and economically due to a lack of qualitative and quantitative performance data. The implementation of the recommendations contained in this report would demonstrate due regard for efficiency and economy of operations, support CNSC management decision-making, and should enable assessments of operations.
Background
The Audit of Contracting and Procurement was included in the approved CNSC Risk-Based Audit Plan: 2008-2009 to 2010-2011.
Contracting and procurement activities in the Government of Canada are governed by a legislative and policy framework which includes the Financial Administration Act (FAA), the Government Contracts Regulations, the Treasury Board of Canada Secretariat (TBS) Contracting Policy and the TBS Guidelines on the Proactive Disclosure of Contracts. The Canadian Nuclear Safety Commission (CNSC) is designated a departmental corporation by Schedule II of the FAA and, as such, is subject to the Government of Canada’s legislative and policy framework for contracting and procurement.
The Government has created a Management Accountability Framework (MAF) which sets out the expectations for good public service management. The TBS regularly conducts MAF assessments of departments and agencies. One of the ten elements of MAF is Stewardship. A specified key indicator of good stewardship is Effective Procurement – “A balanced approach to effective, efficient and economical procurement contributes to valuable program delivery and achieving government-wide objectives.” The key question asked in MAF assessments is does the organization effectively manage procurement.
Contracting and procurement activities in the CNSC are managed by the Contract Management Services (CMS) unit of the Finance and Administration Directorate. CMS policies, practices and procedures are aligned with Government legislation and policy requirements. CMS utilizes the range of procurement methods and tools available from the Department of Public Works and Government Services. Goods and services are acquired by the CNSC using a mix of competitive and non-competitive contracts, standing offers, and local purchase orders. Contracted professional services are an integral part of CNSC operations. In the calendar year 2008, CNSC awarded 1564 contracts of various types with a total value of $22,710,850.
Contracts, January 1-December 31, 2008 |
|
|
|
Category |
Contracts |
Value |
Total Values |
Competitive <25,000 |
5 |
$83,112 |
|
Competitive 25,000-200,000 |
38 |
$3,072,922 |
|
Competitive 200,000+ |
5 |
$1,712,927 |
$4,868,961 |
Non-competitive <25,000 |
233 |
$2,584,932 |
|
Non-competitive >25,000 |
15 |
$658,285 |
$3,243,217 |
Local Purchase Orders <5,000 |
732 |
$895,557 |
|
Local Purchase Orders >5,000 |
147 |
$2,247,457 |
$3,143,014 |
Standing Offers <5,000 |
144 |
$249,423 |
|
Standing Offers >5,000 |
167 |
$8,059,584 |
$8,309,007 |
Temporary Help <25,000 |
35 |
$402,565 |
|
Temporary Help >25,000 |
43 |
$2,744,086 |
$3,146,651 |
|
1564 |
|
$22,710,850 |
Definitions:
Competitive contract – the process used for the solicitation of bids enhances access, competition and fairness and assures that a reasonable and representative number of suppliers is given an opportunity to bid (e.g. MERX, ACAN)
Non-competitive contract – any contract for which bids were not solicited or, if bids were solicited, the conditions of a competitive contract were not met. The Government Contracts Regulations permit the contracting authority to enter into contracts without competition where:
- the need is one of pressing emergency in which delay would be injurious to the public interest;
- the estimated expenditure does not exceed $25,000,
- the nature of the work is such that it would not be in the public interest to solicit bids; or
- only one person or firm is capable of performing the contract.
Standing Offer – an offer from a potential supplier to supply goods, services or both, on the pricing basis and under the terms and conditions stated in the standing offer arranged through PWGSC
Local Purchase Order – non-competitive requisitions used for purchases of goods and services (e.g. coaching sessions, re-upholstering furniture) up to a maximum of $25,000
Focus of the Audit
Objectives
The objectives of the audit were:
- to provide assurance to the President and Senior Management that CNSC contracting and procurement function is operating efficiently and economically with effective controls; and
- to determine if CNSC contracting and procurement practices are compliant with CNSC and federal government policies, statutes and regulations.
Scope
The scope of the audit included examinations of:
- controls established by the CNSC for contracting and procurement,
- selected procurement and contracting transactions with external suppliers focusing on higher risk transactions in the period April 1, 2008 to December 31, 2008, and
- contracting systems, processes and selected contract files for compliance with federal government and CNSC statutes, regulations and policies.
The audit sample included a review of 82 contracts with a value of $3,372,000 as follows:
| Audit Sample Sample Period: April 1, 2008- December 31, 2008 Total contract value for the period $14,593,000 |
|||||||
Number of contracts audited |
|
Value of contracts audited ($000) |
|||||
Type |
Population |
Sample |
% audited |
|
Population |
Sample |
% audited |
Competitive |
43 |
9 |
21% |
|
$4,237 |
$1,680 |
40% |
Non-Competitive |
179 |
50 |
28% |
|
2,101 |
920 |
44% |
LPO>$5K |
67 |
12 |
18% |
|
760 |
215 |
28% |
Standing Offer>$5K |
107 |
7 |
7% |
|
5,448 |
413 |
8% |
Temp Help |
53 |
4 |
8% |
|
2,047 |
144 |
7% |
Findings and Recommendations
Legislative authority to enter into contracts has been delegated to the CNSC. The audit found no examples of non-compliance in the use of delegated financial authorities under sections 32, 33 or 34 of the Financial Administration Act. The CNSC has developed an internal Contracting Policy which respects requirements of the TB Contracting Policy. The organizational structure and roles and responsibilities for contracting activities in the CNSC are clearly defined and documented in corporate documents. Adequate information on contracting and procurement processes is readily available to CNSC managers. Internal communications within the CMS unit are effective; a training program for CMS staff has been developed. CMS utilizes existing government procurement and contracting mechanisms whenever possible. CNSC practices for awarding contracts conform to practices recommended by TB policy.
The audit identified opportunities for improvement in five areas in the management of contracting and procurement in the CNSC.
Finding 1: Monitoring and Reporting
CNSC complies with the TBS Contracting Policy requirements for proactive disclosure. The policy requires public disclosure of contracts and contract amendments in excess of $10,000 on a quarterly basis. In accord with the policy, the CNSC submits regular quarterly reports to the Treasury Board, and publishes the lists of contracts and amendments on the CNSC internet site.
The audit expected to find evidence of formalized monitoring of the contracting function such as quality assurance processes and procedures, supervisor review of officer files, peer file reviews, and regular reporting to management on levels and types of contracting activities.
The audit found that administrative procedures in the contracting function do not include formalized, documented quality assurance steps. The audit examined a sample of contract files from the period April to December 2008. In 22% of the files sampled, administrative errors or omissions were noted. Errors or omissions included things such as statements of work, justifications for amendments or required approvals not present in the file, duplications and coding errors. Individually, the effect of each of the errors or omissions noted was not material or significant, however the risk remains that a material or significant error or omission may occur. The number of exceptions noted indicates that current quality control practices in the contracting function are not sufficient to mitigate inherent risks.
Attention to administrative policies and procedures reduces risks for the CNSC and for CMS. Policies and procedures form part of the overall risk mitigation strategy for CNSC for contracting activities. CMS staff should adequately document their actions and decisions to ensure transparency and accountability in contracting activities.
The audit found that reporting on contracting and procurement activities consists of quarterly external reporting to meet proactive disclosure requirements. Activity reporting to program and branch management, and to senior CNSC management, is ad hoc and infrequent. A monitoring and reporting framework would be expected to include information reported on levels and types of contracting and procurement activities on a regular basis. There are no regular, formalized reporting mechanisms to support management decision-making.
The proposed Contract Review Committee is intended to serve as the primary channel for reporting contracting activities to CNSC senior management. The proposed terms of reference for the committee include guidelines for procurement and contracting reporting to management.
Recommendation
CMS should ensure that effective quality control mechanisms are implemented to ensure that contract files are processed in accord with policies and procedures and that actions and decisions are adequately documented. This should include specifying information to be included in contract files, and regular quality control reviews of files using a risk-based approach.
Management reporting mechanisms envisioned in the proposed Contract Review Committee terms of reference should be designed and implemented.
Finding 2: Performance Measurement
The audit expected to find qualitative and quantitative performance information to demonstrate efficiency, effectiveness and economy of contracting and procurement activities. We expected to find information and data such as:
- contracting and procurement objectives linked to CNSC strategic objectives;
- productivity data, quantity and quality of outputs;
- performance indicators, measures and reports;
- service levels and standards;
- costs;
- benchmarks, standards.
Some qualitative evidence of the level of service and quality of outputs from CMS was found in interviews with internal clients of CMS:
- majority of clients report overall satisfaction with CMS services in interviews
- no concerns were expressed concerning reliability, accuracy, or timeliness of service; and
- there were no indications that CMS and contracting processes unnecessarily impede the conduct of CNSC business.
The audit found that objectives for contracting and procurement activities have not been specified. CMS does not survey client satisfaction.
The audit did not find sufficient quantitative evidence to demonstrate efficiency, effectiveness or economy of contracting and procurement activities. The audit expected to find performance indicators and measures such as times and costs to process various types of contracts, staff productivity measures, and client satisfaction surveys.
- CMS does not collect sufficient, readily available data on productivity or costs;
- Performance indicators and measures have not been identified for the program;
- There are no performance reports to FAD, CSB or CNSC management; and
- Limited service standards have been established for contracting services. The established standards are not sufficient to adequately assess efficiency, effectiveness and economy. Adequate benchmarks or standards for productivity, costs, timeliness, etc. have not been established.
Recommendation
A formal, documented performance measurement framework should be developed to enable assessments of the efficiency and effectiveness of the contracting function, provide reliable information on contracting activities, and support management decision-making processes. The framework should include identifiable business objectives and outcomes; sufficient, adequate performance indicators; methods for collecting performance data; and mechanisms for monitoring and reporting performance information.
Finding 3: Contract Review Committee and Post-Contract Evaluation
The Treasury Board Contracting Policy recommends that contracting authorities establish a formal review mechanism for contracting activities. The CNSC established a pilot Contract Review Committee (CRC) to address the policy recommendation; however the CRC has been inactive since early 2008.
Terms of reference (ToR) for a permanent CRC, including lessons learned from the pilot CRC project, have been prepared and are awaiting approval. The ToR clarify reporting relationships, mandate, membership, contracting review criteria, and roles and responsibilities. The CRC ToR would be supported by a proposed Risk-Based Approach for Procurement and Contracting (see finding #4 for further discussion).
An active, effective CRC would serve an important oversight and challenge function, ensuring that contracting risks are managed appropriately, and play a key role in management reporting on contracting activities.
TB Contracting Policy recommends that contracting authorities should conduct post-contract evaluations of work performed.Section 16.11.8 of the TB contracting policy states ”On completion of the contract, the contracting authority should evaluate the work performed by the consultant or professional.”
The evaluation should include information such as quality of the work performed, efficiency of the consultant or professional in managing time and resources, an analysis of the cost of the work and the estimated value received, contract management; etc.
CNSC does not have a formal post-contract evaluation process. The audit found that some managers conducted informal evaluations on their own initiative to assess the quality of services received.
Recommendation
The CNSC should implement the proposed CRC to fulfill the TB policy recommendation of a formal review mechanism and to ensure effective oversight of contracting activities.
CNSC should implement a formal post-contract evaluation process in line with the recommendation of the TB Contracting Policy to ensure best value in contracts. This could be addressed in the mandate for the CRC.
Finding 4: Risk Management
The audit found that a formalized, documented risk management framework for contracting and procurement activities has not yet been established. There were no documented risk assessments at the time of the audit. Risk management at the time of the audit consisted largely of reliance on existing procedures, the use of Government procurement vehicles and delegated financial signing authority limits. Risks associated with individual transactions were managed informally by CMS staff and CNSC managers.
To address the need for a more formal, comprehensive approach to managing risk in contracting, Contract Management Services has drafted a Risk-Based Approach (RBA) for Procurement and Contracting. The RBA is part of a proposed comprehensive CNSC approach to risk management in contracting in conjunction with a Contract Review Committee. As well, the RBA is intended to ensure enhanced compliance with Government policy on risk management with respect to contracting activities.
Recommendation
CNSC should implement a formal, documented risk management framework for high risk contracting activities. Management should ensure that the proposed Risk-Based Approach includes sufficient procedures to identify, assess, mitigate, monitor and report risks.
Finding 5: Management Fundamentals Training
The TB Contracting Policy recognizes that contract amendments are sometimes necessary and prudent; however the policy recommends that amendments be kept to a minimum.
In the audit, a high number of contract amendments were noted; 36 amendments of various types were noted in 82 contract files sampled. Of the 36 amendments half of these were for extensions or changes to wording of the contracts and did not have any monetary value. The other half were changes to scope that did have a monetary impact; however, the amounts were immaterial. Amendments are part of doing business but if they were not as frequent the CNSC may realize greater efficiencies of the contracting function. Corporate training or awareness of the contracting function may decrease the need for amendments. There is presently no formal, prescribed training for CNSC managers on contracting and procurement practices, planning and management. A module on contracting is expected to be added to the Management Fundamentals training program which is intended to be delivered to all CNSC managers.
Contract amendments can be expected to result in increased administrative costs and to impact the efficiency of the contracting function.
Recommendation
The Corporate Services Branch should ensure that a module on contracting that adequately addresses procurement planning, requirements definition and contract management is included in the Management Fundamentals training program.
Conclusion
Management controls have been effective in ensuring compliance with legislative requirements for contracting activities. The audit found no examples of non-compliance by the CNSC with the legislative and policy framework of the Government of Canada for contracting and procurement activities.
The CNSC has in place some key elements of an effective management control framework for contracting and procurement activities:
- CNSC policies and procedures are generally consistent with Government legislation and policy requirements;
- roles and responsibilities for CMS staff and for CNSC managers are clearly defined;
- sufficient information on contracting processes and procedures is readily available to employees;
- internal communications in the CMS unit is effective, and;
- training for CMS staff is sufficient.
The audit identified opportunities for improvement in the management and oversight of contracting and procurement activities:
- monitoring and reporting of contracting activity should be improved to ensure transparency and accountability;
- a performance measurement framework should be developed and implemented to demonstrate due regard for efficiency, effectiveness and economy of contracting activities;
- a contract review committee should be implemented to ensure effective oversight and challenge of contracting activities;
- a post-contract evaluation process should be implemented to ensure best value in contracts;
- a formal, documented risk management framework, such as the proposed Risk-Based Approach for Procurement and Contracting, should be implemented to ensure that risks associated with high risk contracting activities are identified, assessed, mitigated, monitored and reported;
- training on procurement planning, requirements definition and contract management should be required for all CNSC managers.
There is insufficient performance information available to conclude whether CNSC contracting and procurement activities are operating efficiently and economically. The implementation of the recommendations contained in this report, particularly the recommendation for development and implementation of a performance measurement framework, would demonstrate due regard for efficiency and economy of operations, support CNSC management decision-making, and should enable assessments of operations.
