Procurement - Assessment of Agency's Capacity

Introduction

1. Carrying out procurement efficiently under Bank-financed projects is critical to good project implementation, to the attainment of the objectives of the projects and to their sustainability. Equally, the Bank, as part of its developmental role, is interested in strengthening the capacity of its borrowers to administer procurement in an effective and transparent way as part of sound governance and good project management. In this context, project teams are required as an integral part of project preparation and appraisal, to make an assessment of the capacity of the project implementing agency or project implementation unit to administer procurement.[1] This more general procurement capacity assessment (not necessarily for Bank-financed procurement) is of paramount importance in programmatic approaches, including joint assessments with other donors. For specific details on SWAps, see Annex B of the Fiduciary Arrangements for Sectorwide Approaches (SWAps). The objectives of the capacity assessment are to:

(a) evaluate the capability of the implementing agency and the adequacy of procurement and related systems in place, to administer procurement[2] in general (especially in the case of SWAps) and Bank-financed procurement in particular;

(b) assess the risks (institutional, political, organizational, procedural, etc.) that may negatively affect the ability of the agency to carry out the procurement process;

(c) develop an action plan to be implemented as part of the project, as necessary, to address the deficiencies detected by the capacity analysis and to minimize the risks identified by the risk analysis; and

(d) propose a suitable Bank procurement supervision plan for the project considering the relative strengths, weaknesses and risks revealed by the assessment.

Preparation of the Capacity Assessment

2. The capacity assessment should be carried out as part of the project preparation and it should be carried out by a Procurement Specialist (PS) or a Procurement Accredited Staff (PAS) assigned to the project at PCD stage. The aim is to have the assessment and the agreed action plan finalized at the time of appraisal. The nature of the project would determine the specific areas that require more in-depth analysis at capacity assessment. For instance, a SWAp assessment would address sectoral issues, as well issues raised in individual capacity assessments carried out for projects in that sector. On the other hand, the capacity assessment for a Learning and Innovation operation would be simplified. The Regional Procurement Adviser (RPA) can advise on the extent and scope of an assessment. The assessment and development of the action plan should be carried out with the full participation of the borrower and of the implementing agency to ensure ownership of the proposed actions. The actions proposed in the plan should be incorporated as project components and funded through the loan/credit as needed. The analysis may detect critical deficiencies that need to be addressed before project launch (i.e., setting up of a procurement unit or preparation of critical bidding or contracting documentation). In the event that the implementing agency cannot finance these activities, the project team should discuss and agree with the borrower alternative financing sources (i.e. a PPF, retroactive financing of these activities, etc.). For projects involving several implementing agencies, the project team should assess the capacity of the agency nominated as the lead or coordinator for the project, and all those having a major role in procurement administration for the project, or a representative sample of agencies if all play a similar role. This assessment, as in the case of a single agency, will be the basis for designing the capacity strengthening and procurement supervision plans.

3. The PS or PAS assigned to the project is primarily responsible for carrying out and coordination of the assessment. Those aspects of the assessment that deal with the controls in place (financial and administrative), should be carried out with the help of the disbursement or financial management specialist (FMS) assigned to the project. This approach is necessary due to the specialized nature of assessing financial controls and because an assessment is also required for Financial Management purposes. The Legal Department should be involved as necessary in relation to any issues related to legal aspects of the procurement.

Scope of the Review

4. The capacity review includes an assessment of the capacity of the agency to carry out the different phases of procurement. The assessment should look into the organizational aspects, skills of the staff, quality and adequacy of supporting and control systems, and suitability of the laws, rules and regulations applicable to the agency. Appendix 1 to this memorandum provides a checklist in the form of suggested questions to help in the preparation of the assessment. The purpose of this list is to guide the assessment and there is no need to answer those questions or to request that the implementing agency answer them -- this questionnaire is not mandatory. The list is provided as a guide for the assessment and should be applied flexibly depending on each case. For example, for new borrowers or implementing agencies it may be necessary to go over the entire range of items listed in Appendix 1. However, if the borrower being assessed is well known to the Bank, much of the information listed in Appendix 1 is likely to be available in the Bank and the majority of the assessment can be done without consultation, leaving only critical parts of the assessment to be updated or carried out in the field, if necessary. Again, the RPA may help in determining the scope of the assessment. Equally, if there is an up-to-date CPAR, that report should have addressed several of the questions in the questionnaire. Typically, the assessment includes a review of the following aspects:

(a) Legal Aspects and Procurement Practices. This aspect of the review should consist of (a) a quick verification to ensure that the applicable procurement laws and regulations will enable the institution to carry out acceptable procurement; and

(b) an assessment of the quality of internal procurement practices of the implementing agency or agencies and their conformity with procurement practices acceptable to the Bank. The latest Country Procurement Assessment Review (CPAR) should be consulted for relevant information in regard to general country rules and environment that may be relevant to the project. Specific items to look into are:

(b) Procurement Cycle Management. This item includes a review of the general quality and timeliness with which the agency or institution handles each phase of the procurement cycle. The key elements are:

(c) Organization and Functions. This is a review of the organizational structure of the procurement unit, how responsibilities are allocated, its reporting relationships, its decision-making authority and whether it has the capacity to handle the proposed procurement plan for the project in addition to its other routine duties if any. Specific items to review are:

(d) Support and Control Systems. This item deals with services and control mechanisms that provide checks and balances in the system. The independence and credibility of procurement audits and the quality of internal controls are critical to the reliability of the system. Specific items to be considered are:

(i) Procurement oversight and Auditing
(ii) Internal technical and administrative controls
(iii) Code of professional behavior and ethics
(iv) Special anticorruption initiatives

(e) Record-keeping. The team should take particular note of the availability, quality, security and completeness of procurement records and files. In addition to overall data on numbers, types, values and dates of contracts awarded and names of awardees, procuring organizations should maintain for all contracts, a record which includes, inter alia:

(i) public notices of bidding opportunities
(ii) bidding documents and addenda
(iii) bid opening information
(iv) bid evaluation reports
(v) formal appeals by bidders and outcomes
(vi) signed contract documents and addenda and amendments
(vii) records on claims and dispute resolutions
(viii) records of time taken to complete key steps in the process
(ix) comprehensive disbursement data (as required by the country's financial management system)

The absence of or significant deficiencies in such records should be reported in the assessment findings and addressed in the recommended action program.[4] The team should also determine to what extent effective procurement monitoring systems are in use and, if sufficient data exists, it should identify steps in the procurement process where inefficiencies seem to exist and recommend ways bottlenecks might be eliminated.

(f) Staffing. The quality and quantity of the staff the unit are essential to good procurement administration. The assessment should determine in general whether sufficient qualified staff are available to carry out the normal procurement tasks assigned to them. There should be a determination whether the existing staff have relevant knowledge of the disciplines and the capacity required for carrying out the proposed procurement plan under the project. Otherwise, the assessment should define the assistance required in the form of training additional staff or consultants or procurement agents, and include an estimate of the scope, duration and cost of these services and additional resources.

(g) General Procurement Environment. It is also necessary to examine the actual performance of the procurement unit as evidenced by whether timely decisions are taken, how often contract award decisions are protested or overturned, whether adequate records are maintained and similar indicators, and to try to identify the underlying causes for any areas of bad performance. Poor procurement quality often results from underlying factors inherent in the society or in the organization carrying out procurement. Such factors include:

(i) the degree to which high levels in the government promote a culture of accountability
(ii) the reputation of the procurement entities/personnel
(iii) the salary structure of procurement staff versus comparable salaries in the private sector
(iv) the degree to which the procurement unit and the institution are free from political or other interference
(v) the existence of capable procurement staff
(vi) the presence of clear written standards, procedures and delegations of authority and responsibility
(vii) the soundness of the agency's budgetary and financial management systems, etc.

(h) Private Sector Viewpoint. An important part of assessment is the views of private firms dealing with the agency on how the written regulations and procedures are applied in practice. In this respect the team conducting the assessment should contact private firms, as appropriate, to find out their views on aspects such as:

Risk Assessment

5. A key aspect of the assessment is the analysis of risks in the procurement process. This analysis should include the risks in all areas covered by the assessment and is closely related to the transparency and predictability of the agency's implementation of the procurement process. The review should look at the record of the institution in handling procurement in general and Bank-financed procurement in particular. The CPAR, which addressees the general risk environment in the country may be of guidance. Of particular relevance is the consistency of application of the written rules and procedures in practice. A key input in this review is the opinion and perception of private sector parties, especially consultants, suppliers and contractors working for the agency, both national and foreign. Key actors in the private sector business community dealing with the institution should be interviewed as part of the risk assessment. This assessment should culminate with a rating of the procurement risks as high, average, or low and is key in the determination of the supervision approach to be recommended. Appendix 2 provides a form to be used as a working tool to evaluate the risk in each of the findings of the review and to facilitate the overall assessment of risk.

6. The risk assessment requires considerable professional judgment and there are no precise instructions or steps leading to a clear cut overall assessment. Moreover, different institutions may present weaknesses in the same areas but with varying degrees of severity. In general, an institution showing severe deficiencies in key areas covered by the assessment should fall into the high risk category, and one showing satisfactory ratings in most of them should be in the average risk zone. Only those showing good or better ratings in all areas should be rated as low risk.

Action Plan to Build Agency's Capacity

7. The analysis described in the earlier paragraphs should identify the actions to be taken and the associated timetable to improve the long-term capacity of the agency to administer procurement. Actions may cover matters such as exceptions to local law in the loan agreement (those exceptions indicated in the CPAR for the country), processes, staffing, organization, training, record keeping, auditing, etc. Those that are essential for project implementation should be in place before procurement starts. Others may be implemented during the life of the project. The timetable proposed should therefore reflect these priorities. The plan should be developed in detail, including TORs for any consulting assignments and cost estimates. The detailed plan should be part of the project implementation documentation and should be agreed to with the borrower during negotiations as a project component.

8. The intensity and nature of the supervision of procurement is linked to the capacity of the institution through the setting of prior-review thresholds and the extent and depth of the post-reviews and audits to be carried out. The supervision plan is also determined by the nature of the project (whether it is a project involving a few large contracts or many small or medium-sized ones or whether it is implemented centrally or in a decentralized way). The ability of the agency to finance a contract itself if the Bank needs to declare misprocurement and cancel part of the loan, is another factor to consider. Finally, the assessment of the procurement risks (see para. 5) should be a key factor in deciding on the supervision plan (as per para. 10, below).

9. Agencies assessed in the high risk category should not have prior-review thresholds for procurement of goods above $ 250,000, $ 500,000 for works, $100,000 for consulting services with firms and $50,000 with individual consultants; agencies with average risk should not have thresholds exceeding $500,000 for goods, $5 million for works, $ 350,000 for consulting services with firms and $100,000 with individual consultants; and, those in the low-risk segment should not have the thresholds exceeding $ 1,000,000 for goods, $10 million for works, $750,000 for consulting services with firms and $200,000 with individual consultants.In all cases of single-source selection (contracts with firms or with individual consultants) the Bank should give the "no objection" to the proposed assignment. For loans through financial intermediaries, the prior review should cover all contracts awarded under ICB as a minimum. These thresholds are ceilings not to be exceeded without prior OPCPR clearance, but RPAs may decide to set lower thresholds for example in cases of very high risk agencies or countries, when the project requires intensive review or when the agency's capacity is only marginally into one of the medium- or low-risk categories. The long-term objective is to strengthen the capacity of all agencies to move them gradually into the low risk category. The legal agreements may include, if appropriate, the possibility of a gradual raising of the threshold as the agency meets specific milestones of the strengthening program as an incentive. The agreements should also include the possibility for the Bank to reduce the threshold, increase the intensity of post-reviews and/or launch an in-depth procurement review or audit, if the capacity of the agency deteriorates or if a pattern of problems emerges from its performance or from the results of post-reviews.

10. The supervision plan should include a recommendation on the frequency of post-review missions. The number of those post reviews should be related to the capacity of the implementing agency, but should not in any case be less than every six months in high-risk agencies, every twelve months for average-risk agencies and every eighteen months for low-risk agencies. This should be supplemented by an appropriate independent post review depending on the factors mentioned in para. 8.

11. The size of the sample for post-review should consider factors similar to those mentioned in para. 8. Generally not less than 1 in 5 contracts should be reviewed for high risk agencies, 1 in 10 for average risk and 1 in 20 for low risk ones. As in the case of prior review, the ratio may be adjusted during project implementation depending on the performance of the agency and the results of the reviews. The procurement supervision plan should recommend that independent post reviews of procurement be carried out, if appropriate, giving the number and scope of such reviews, and propose whether the supervision plan should include special missions for procurement supervision at critical points of project implementation.

12. The procurement specialist may discuss the assessment and the proposed supervision plan with the RPA, as needed during preparation, to ensure consistency throughout the region. The RPA clear the plan as part of the clearance of the final project package before negotiations.

13. Follow-up actions: When the capacity assessment recommends actions to be implemented to improve the PIU procurement capacity as part of the project implementation, it is recommended that follow-up of its implementation be included in the supervision plan. This follow-up should result in stronger supervision at the beginning of project implementation, with verification of initial compliance during the project-launch mission and an immediate verification not later than six months after project start.

14. For keeping records of the conclusions and recommendations of the assessment, the procurement specialist prepares a summary report and submits to the RPA for clearance. The report should contain the proposed supervision plan (including the thresholds for prior review and frequency of post-review missions). This summary report is the sole report required (it is not necessary to complete Attachments 1 and 2 of this memorandum). The report should contain as minimum:

[1] The assessment of procurement capacity at country level is addressed through the Country Procurement Assessment Report (CPAR) under a separate instruction issued on May 23, 2002.

[2] Includes goods, works, services and consulting services, as applicable. Therefore the assessment should include a review of the administration of selection and employment of consultants process.

[3] Refer to the Instructions on how to carry out CPAR for acceptability of NCB practices.

[4] The critical records that shall be properly maintained (on the procurement process) up to two years after the loan closing date are: copies of all public advertisements, pre-qualification documents (if used), the pre-qualification evaluation report documenting any decisions not to pre-qualify certain potential bidders, the bidding documents and any addenda, a record of any pre-bid meetings, the bid opening minutes, the final bid evaluation report (including a detailed record of the reasons used to accept or reject each bid), copies of bids, appeals against procedures or award recommendations, a signed copy of the final contract and any performance and advance payment securities issued.

Shortcuts

Assessment of Agency's Capacity