Official Report

Agenda item 4 is a section 23 report. I welcome our witnesses, who are from the Scottish Government. We have Leslie Evans, the permanent secretary; Alyson Stafford, director general, Scottish exchequer; Gordon Wales, chief financial officer; and Colin Cook, director of digital. I understand that the permanent secretary has an opening statement.

The Scottish Public Pensions Agency provides pension administration services to more than 0.5 million retired and contributing members across the national health service, teachers, the fire and police services and the Scottish Parliament. The agency has experienced material growth in the number of scheme members over recent years, including successfully taking responsibility for the police and firefighter schemes in 2015. In addition, the agency has implemented a range of information technology and change projects to respond to UK and Scottish pension reforms and system improvements.

The agency’s transformation programme was launched six years ago, in 2013, and comprised eight major projects. All bar one were implemented successfully. The decision to terminate the one unsuccessful project—the integration of pension administration and payment functions, which is the subject of our deliberations today—was taken almost two years ago in February 2018. The decision was taken only after months of strenuous efforts on the part of the agency to secure a working system from the contracted IT software supplier, Capita. Despite repeated reviews and replannings, Capita failed to deliver any of the project milestones.

In the committee’s previous evidence session on the report, you heard that the SPPA has undoubtedly extracted and applied important learning from the experience, and the Scottish Government has learned, too. We have put in place more robust procedures for the progress of IT projects and we are strengthening and clarifying relationships with arm’s-length bodies. Procurement regulations have also tightened in relation to areas such as low tenders. However, as Audit Scotland’s report confirms, Capita’s delivery of the software was fundamental to the success of the project.

I stress that, throughout the period under discussion, the SPPA maintained collection, administration and payment to clients and continues to do so. It is important to emphasise that there has been no risk to the pensions that are paid by the agency. They continue to be paid on time and in full. The focus has been and will always be putting customers first.

Audit Scotland’s report sets out costs for the life of the project but, in practice, the additional costs to the SPPA were smaller and were closer to £3.8 million. That includes significant sums on existing staff who would have been employed anyway, as well as costs to Capita. Cost projections for the future are also very different and are dependent on the pace of change.

I am aware that, over the years, the committee has considered a number of public sector IT projects. The lessons from those projects have shaped current practices and new Scottish Government assurance arrangements. Those lessons have also enabled Audit Scotland to develop helpful guidance for digital programmes right across the public sector. However, it is worth noting that we currently have 410 active IT projects registered with our digital assurance team, 27 of which are classed as major. In the past year alone, the Scottish Government and its agencies have successfully delivered several new IT projects and systems, including the upgrade to Revenue Scotland’s tax and administrative system, several new benefits in Social Security Scotland, improvements to our core shared service platform and the creation of two new forestry agencies.

The chief executive of the SPPA, as the accountable officer answerable to the Parliament, will always be best placed to answer detailed questions on the issue. You will be aware that much of the information and paperwork pertaining to the project is archived, so we may have to come back to you on particular points of detail. However, I am here today with my colleagues to assist the committee in its scrutiny of the project.

Thank you. As you mentioned, we had an evidence session on the issue, but the committee was not satisfied with the answers that we received, which is why we have invited you along, as the person in charge of the Scottish Government. The chief executive of the SPPA is new and so did not have detailed knowledge of what went on previously. We hope that we can get clarity on some of the issues today.

Ms Evans, you will be well aware that a number of IT projects have come to the committee’s attention. We tend to ask the same questions every time that we get those reports, because there is always a similarity in them with regard to a failure to adopt recognised software development and quality assurance management standards and practices. It seems that this might be another example of that. Why did the failures take place in this project? Were those recognised management standards adopted and in place for the project?

Colin Cook might want to say something about the technical aspects of the software.

It is important to remember that the SPPA has its own board and accountable officer and is responsible as a decision-making body in its own right. It was clear from the beginning about its roles and responsibilities and what it was seeking.

We can talk a little about the support that we gave the SPPA as it followed the very clear procurement process, which, at the time, was governed by the Public Contracts (Scotland) Regulations 2012. The regulations have since changed, so the process by which the software would be procured now would be quite different.

The other thing to be clear about is that the SPPA received support from the Scottish Government’s procurement section and lawyers, but its decision-making process was based on rigorous testing of the Capita tender. A number of companies came forward in the procurement process on the basis of the descriptions of what was required that were put out. In those days, there was a 70:30 split in terms of quality and price, and Capita scored well on that.

Perhaps Colin Cook could say more about the specifics and a little about how the process would be different if we were looking to go through—as we are—the same exercise now.

We have discussed on a few occasions the new principles and approach that, with Audit Scotland’s support and guidance, we have introduced across the Scottish Government. Such a major project would now need to be registered with the internal audit team that oversees digital projects, and it would be subject to mandatory justification gates throughout its lifetime. For the people running the project, there would be a set of guidance available and a set of resources for them to call on. There would also be a set of standards, known as the digital first standards, which summarise need, to ensure that user needs are well understood before the project starts; the technology that should be used, or the approach to procuring or developing that technology; and the capability and capacity that the team will need to develop the project and sustainably improve any service that emerges after it goes live.

Thank you.

Software development methodologies have been around for ages and are fairly well established in the industry. However, this project seems to be an example of those methodologies not applying—although they now will. Given that standards have been in place for so long, committee members have been asking for a number of years how that can happen with projects such as this one. It is as though the project slipped under the radar and we did not notice it. I think that that is a common feeling among members. When we suddenly get such a failure, we hear people say, “Oh, we are applying new standards now”, but the standards have been around for a long time. Why were they not applied in this case?

I will say one thing before Colin Cook comes in on the specifics.

The other thing that we need to remember is that we had a bid from a contractor that is very experienced in not just software but pensions, and which still delivers pensions contracts in the UK and, possibly, beyond. The bid was tested robustly; we might want to talk more about that process. However, on robustly and rigorously pressing Capita’s preparedness and capacity to deliver the project, the then chief executive was very experienced and took timely action to take extra time to test Capita’s capacity, experience and pricing of the bid, which was not necessary under the procurement regulations of the time.

Although I understand your frustration with particular software projects—Colin Cook can talk a bit more about how we are evolving and have developed our approach—there is a contractual compliance issue here that we should not forget. We may want to talk more about that later.

Permanent secretary, that is an interesting response. The Auditor General reported that the SPPA’s response was that

“it did not have the skills to further probe the tender.”

That is quite contrary to what you just said.

The SPPA was looking to probe the tender across a range of areas. There is a bit of confusion about the exchange with, I think, the legal advisers in the Scottish Government—

The exchange between—

The SPPA.

And Capita?

No. There was an exchange—indeed, there were many exchanges—between the SPPA and Capita, but if I remember the report correctly, there was a discussion about when SPPA felt that it was appropriate to keep pursuing Capita, and along what line. My assertion here, and the evidence that we have—Alyson Stafford and others, probably including Gordon Wales, have seen the raw material on all this—is that the chief executive not only sought legal advice but asked for written clarification from Capita about its costings; how many data migrations it had experience of across public and private sector schemes; its ability to deliver economies of scale; and the view that it wanted to grow its market in Scotland and that that was one reason why its tenders were as they were. The SPPA also wanted to know what it would need to do if it felt that the tender was too low—in other words, that it was not robust enough. That was part of the conversation that I believe it undertook with experts in the Scottish Government.

Okay, but are you suggesting that the contract was probed further?

The tender, yes.

Sorry—the tender.

The Auditor General said that the SPPA itself responded that

“it did not have the skills to further probe the tender.”

That indicates that no further probing was done. The report goes on to indicate that the SPPA “took no further action” before accepting Capita’s bid.

The SPPA tested the tender rigorously. I know that it also had some specific technical issues that it wanted to consider with Capita; I think that that was the conversation that we are talking about. At no time were we asked for any further support by the SPPA on probing the tender, because the SPPA was so fundamentally—and characteristically, I think, given the chief executive’s experience—thorough in its testing. I am not aware of any aspect of the tender, or the probing of the tender, that would have made a difference to Capita’s fulfilling the project.

So why did SPPA say to the Auditor General that

“it did not have the skills to further probe the tender”?

Are you saying that it did not need to probe the tender further—that it was all done?

I think that it probed the tender very seriously, and more than one might have expected, given the regulations that were in place at the time. I think that if it felt that it needed to gather more information, or more experience, it could have come to the Scottish Government for that. I am not aware of whether it did or did not do that. Do colleagues want to come in on that?

I am very happy to do so. The key thing about the history of the tender and Capita’s involvement is that it started way back in 2014. A pre-qualification questionnaire was issued, to which various suppliers responded. Capita was one of those, and its response showed that there were no grounds for concern, in terms of its finances or its capability to ultimately take part in the tender. Tender bids were then submitted by the due date, which was in March 2015. Capita’s bid, alongside all the other bids that were submitted, went through a rigorous test to rank the position of all the bids that had come in. As the permanent secretary has already said, that process was based 70 per cent on quality and 30 per cent on cost. The key part of it was to work through what is known as the MEAT—most economically advantageous tender—method. The whole package is taken into account.

The agency went through due process and, as part of the process, carried out some further scenario tests, asking each of the bidders to work through three scenarios that involved demonstrating processes—for example, demonstrating how members would apply for benefits online or following a calculation right through, from receiving mail to the final stage. Those detailed things were all tested, and Capita came out on top for all those presentations.

Further checks were done. As the permanent secretary has said, those further checks, with the level of due diligence as described, were not actually required under the tendering regulations that were in place at the time; they were carried out to ensure that the bid was not an abnormally low tender. It was for Capita to demonstrate that the bid was both genuine and serious. It responded on that point, and its representatives were seen by the accountable officer at the agency. The judgment was made that the bid was a genuine and serious bid.

It is important for us to remember that what happens in procurement exercises is a highly regulated but also commercial activity. Companies that compete and take part in a tender exercise will make a whole range of judgments as to how they want to take part in it. Part of the testing of a low bid involves ensuring that the contract requirements are absolutely understood. The contract terms were supported by the Scottish Government’s legal department, in order to ensure that they were rigorously drafted. Capita was able to demonstrate to the accountable officer that its understanding of the contract’s requirements was not weak. We were alive to the fact that a supplier could perhaps choose to exploit the situation: once it had put in a low bid, it could come back during the term of the contract and exploit the situation to try to make good.

There have been some real decisions about tenderers that may wish to get into a market and are willing to put forward a different price. That is not unusual in a commercial scenario. We have had examples in Scotland—not necessarily involving IT—of suppliers and successful tenderers to which we have let contracts and which have put forward their materials with a price that was lower than what might be viewed as an appropriate price so that they could get a market share. Some companies are willing to enter a market just so that they do not have fallow periods or staff who are not being deployed.

We appreciate that the scenario that you have described can happen. I am more concerned about the first part of your answers, which stands in direct contrast with paragraphs 14 and 15 of the Auditor General’s report. You said that Capita came out on top in every instance. The Auditor General said:

“Capita scored second for quality, first for cost, and first overall.”

I agree with the Auditor General’s report in terms of—

You do?

I do in terms of the ranking against quality and cost but, as regards the further tests that were applied to all bidders, with three scenarios that each of them had to present against—

Okay. Well, let me—

That went beyond the 70:30 quality and cost ranking. Even beyond that, some further exercises were done, and they were applied to all bidders at the same time. It was a competitive process. On the basis of those three scenario testings, Capita came out on top.

I think that we are all agreed that its bid was the lowest. However, paragraph 15 of the report stands in direct contrast to what you and the permanent secretary have described to us this morning. I will read it to you:

“As SPPA classed Capita’s bid as abnormally low, it was required to investigate whether it was a valid bid. SPPA queried elements of the bid with Capita and discussed Capita’s responses with the Scottish Government Legal Department, who advised that more in-depth questions should be asked to fully assess the bid. SPPA informed the Scottish Government Legal Department that it did not have the skills to further probe the tender. SPPA took no further action and accepted the bid and reassurances from Capita over its ability to deliver.”

Do you accept the Auditor General’s account?

I know that the SPPA had to determine whether Capita’s bid was genuine and serious—that was the overarching test that it had to do. It conducted due diligence on the bid. Again, that was not required at the time, but it chose to do that—and absolutely did that—as part of following that through. The clarifications that it sought from Capita gave it the assurance that it needed that the bid was genuine and serious.

What is your reaction to the SPPA saying that

“it did not have the skills”

to ask the necessary questions?

Obviously, that is what is on—

Do you accept that?

I see that that is what is on the record, and—

But you do not accept that.

I know that, ultimately, a judgment had to be exercised by the accountable officer on the contracts and the bids. On balance, across all the bids—and the due diligence on Capita was reasonable—the accountable officer saw that all the assurance that they had up to that point was sufficient to make a decision.

I do not think that that issue made a fundamental difference to Capita’s capacity to deliver on the contract. That is an important point. We saw that in the report. Capita failed from the first project milestone to deliver on the project. From very early on, it indicated concerns and difficulties about being able to deliver.

Do you accept that the Government failed to check out Capita properly?

I accept that the chief executive, as the accountable officer, his team and the board did everything that they could. In fact, they did more than was required to check out Capita’s capacity.

They did not, because they said that they

“did not have the skills”

to ask the right questions.

I do not know where that comment came from, but—

It came from the Auditor General.

I mean in terms of the SPPA. Ultimately, it was for the chief executive to probe every aspect that he felt was appropriate to probe in letting the contract, to advise his board on that basis and to decide that the contracted has been tested—not quite to destruction, but the tender was heavily tested; it was tested way over and above what was required for procurement at that time. Are we saying that there is one element that led Capita to fail to fulfil the contract? Capita would need to be asked about that.

But your officials said that they

“did not have the skills”

to ask Capita questions. That is part of the problem.

The SPPA said that it might not have had the skills, as far as I can remember from Audit Scotland’s text. I cannot understand what the context of that comment was.

No, the SPPA did not say that it “might not have had the skills”. The exact wording is:

“SPPA informed the Scottish Government Legal Department that it did not have the skills to further probe the tender.”

I sincerely apologise to Willie Coffey. Would you like to come back in? I know that other colleagues want to probe that issue a little more, too.

I will be brief.

I go back to the software that did not work. My experience from a number of years of working in information technology is that, if people adopt and apply a recognised development standard, they are quite likely to get things right. What development standard was applied in the project? What software development methodology was applied?

In this instance, we have seen from the reviews that took place that there were a weakness in how the program was developed by the contractor and weaknesses in the way in which the SPPA was skilled and in its ability to oversee the development of the project.

I can assure members that, if such a project were starting now, the ways in which the assurance processes would work, the support that we could provide to the SPPA, how it would engage with that support and the standards to which we would operate would insist on and require us to follow a recognised methodology, whether that was a standard waterfall-type project management approach or, as would be more likely in this case, some form of agile project development.

I can also assure members that the training that is required in order to promote those standards is in place. We have launched a skills academy. The Scottish digital academy has trained more than 1,800 people in such processes. We are managing to train the practitioners who are responsible for delivering the projects and the senior managers and directors of organisations, to give them a better understanding of the ways in which software should be developed.

That is a good answer, but it tells me that no software development standard was applied in this project. That is what I am trying to get at. The issue has been known about for a long time, but the project seemed to escape the clutches of recognised development methodologies. Nobody noticed until it was too late, and we lost £8 million as a result—a figure of £8.7 million was mentioned by the Auditor General, £700,000 of which was returned by Capita.

If no standard was applied, and we are correcting that so that we now apply it, we might as well say so. The message that has been sent for a long time is that the adoption of recognised IT development standards must be a priority. I am encouraged by what you have said, because it appears that that standard is now in place. However, it was not in place in this project.

Yes, and I think that you saw from how the reviews took place that there was a willingness at the beginning of the process to try to put the project back on an even keel and start to introduce some of the disciplines that one would expect, and time was given for that to happen. The way in which the review process was carried out set some milestones and anticipated gates that we would expect people to meet. When they were not met, the new—as it was at the time—audit and assessment process was robust enough to say that the project was one that we could no longer support, and the chief executive took the decision on that basis.

I think that I can offer you confidence that we are in a different place now in terms of skills, methodologies and capabilities in the organisation. However, clearly, this project did not work, and, as a consequence, we have learned from it and are moving on.

Sorry to be pedantic, but to come back to paragraph 15, the SPPA clearly had concerns about and queried aspects of the bid. It raised some of Capita’s responses with the Scottish Government’s legal department, which agreed that there was a problem and asked the SPPA to ask more in-depth questions. The SPPA told the legal department that it did not have the skills to do that. What did the legal department do at that point?

I cannot speak on behalf of the legal department in those days. I think that, now, the legal department would probably say, “We have given you our advice. If you need further support or skills sources, there are places in the Scottish Government where those can be sourced, depending on what the specific issue is.” The project was a very technical one, and the advice that was given by the Scottish Government at that time was predominantly to do with procurement and the legal aspects of procurement. It would not necessarily have been the lawyers who would have been able to either spot the issue or answer questions about what to do about it.

If the aspect of the software that was the problem was a particularly technical one—I do not know how technical it was; Gordon Wales might be able to help with that—that would not necessarily be something that the legal department or the procurement department could help with. However, those departments could have sent them elsewhere, such as to Colin Cook’s predecessors in IT and to our technology side of things. However, to my knowledge, we did not receive a bid for that kind of information or support, although we gave masses of support in other areas, and continued to do so throughout the project.

I understand that it is important that you probe the issue in paragraph 15, but if you are asking whether that issue is at the core of why the project failed, I would say that I do not think so.

I think that it is rather more indicative of why the project failed. There is a series of multiple failures throughout. That is just one aspect of it. I would have expected someone in the legal department, when they got that response from the SPPA, to have the intelligence to say that there is a problem here. Did the department just file the issue? It has a corporate responsibility.

Indeed it does, and I would not demur from that. I would like to think that, because we all have a corporate responsibility for ensuring that the Scottish Government and arm’s-length organisations deliver on projects, if something of that nature came up now, it would be dealt with differently. Of course, the decision to accept the tender was the chief executive’s decision, based on all the information that he had at the time from the Scottish Government, in a supporting capacity, and from his own officers.

So, it would be the chief executive who would take the decision to take no additional action.

Yes, that would be his operational responsibility; he would be accountable for that. He would have weighed up all the information that was in front of him, having, as Alyson Stafford described, tested almost to destruction the nature of the tender and having had contact with our procurement and legal sides. On the basis of all that, he would have decided whether he felt that it was the right kind of tender for the project that we described.

Is there documentary evidence of the decision and why it was taken?

I am sure that there will be, because it will have been submitted to the board. I do not have the minutes with me, but we might be able to find those for you.

It would be interesting to see them.

Permanent secretary, forgive me, but I am confused by some of your answers. You say that the chief executive tested the tender to destruction. He did not test it to destruction. It says in paragraph 15 of the Audit Scotland report:

“SPPA queried elements of the bid with Capita and discussed Capita’s responses with the Scottish Government Legal Department, who advised that more in-depth questions should be asked to fully assess the bid.”

The legal department of your Government told the SPPA that it needed to ask more questions. Therefore, the chief executive at the time could never have been under the impression that he had tested the tender to destruction before he decided to accept the bid, so why do you say that?

The chief executive is the accountable officer who is responsible for taking the decision. He—or she, in certain circumstances—has to look at all the information that has been amassed at the various stages of tendering that Alyson Stafford described, the probing that has been undertaken and what came out of that probing. Depending on how technical or financial the tender is, or depending on its quality or presentation and so on, the chief executive has to recommend whether the tender should be accepted.

You said that it had been tested to destruction, but your legal department said that it had not been and that there were additional questions to be asked.

In the context of the regulations and procurement processes of that time, the investigations that the chief executive undertook were not required. Now, they would be, and a bid of this kind would probably not be accepted, although Colin Cook can advise me on that. The regulations have changed; they were quite different several years ago.

But your legal department advised the SPPA that it needed to—it was required to—ask more questions. The Audit Scotland report says that your legal department

“advised that more in-depth questions should be asked to fully assess the bid.”

Perhaps I could interject. The legal department reviewed the Capita’s responses and described them as reasonable. It then suggested that an additional question could be asked. It did not dictate that the agency should ask additional questions. The legal department’s assessment was that the questions that Capita had been asked thus far had elicited reasonable responses.

You are therefore saying that paragraph 15 in the Auditor General’s report is not worded all that accurately.

It might well be that the SPPA was not in a position to ask additional technical questions, but the reality is that the chief executive—

No, paragraph 15 does not say that. It says:

“the Scottish Government Legal Department ... advised that more in-depth questions should be asked”.

The evidence that I have seen says that the legal department suggested that additional questions could be asked.

Paragraph 15 says “should”.

I realise that there is a subtle difference.

There is a difference between “could” and “should”, yes.

Ultimately, as the permanent secretary says, it comes back to whether the accountable officer was satisfied that the probing that had been carried out up to that point was reasonable and satisfied him that Capita’s bid was a serious one. He took the decision that it was.

He took that decision despite the fact that he had been advised that more questions needed to be asked.

They could be asked.

Could be asked.

It says “should” in the Auditor General’s report.

Our understanding is that it was “could” not “should”—we have only a limited amount of information to go on, but we have some that indicates that it was “could”.

The Auditor General is sitting behind you and we will hear what her take on that is during our next session.

Why did the Scottish Government not intervene at this point?

The Scottish Government was in a position to support the accountable officer’s role in testing the tender. The chief executive of the agency would come to us and tell us, which I think he did, that his intention was to accept the tender. A process was then triggered by that decision. I am not familiar with all the details of the process but, to my knowledge, at no time did the chief executive ask the Government to tell him whether he should accept the bid. If that had been the case, we would have had a conversation about it. Alyson Stafford can say whether that was the process.

A specific process is followed for all procurements, which is designed to ensure that diligence is applied to all bids equally. Overall, based on the different scorings on the 70 per cent quality element and the 30 per cent costs element, Capita’s bid came out as the most economically advantageous bid. In the additional tests on the three scenarios, it came out top. Additional diligence was carried out to ensure that the bid was not an abnormally low tender. Capita was asked to provide clarification and assurance that its submitted costs were accurate and sustainable through the term of the contract, and Capita was able to confirm that its bid was accurate and that it allowed for its required margins and contingency. All those factors are the things that the accountable officer at the agency took into account in reaching a judgment.

The other thing that I think is important—

I accept—the Auditor General has made this clear—that the SPPA chief executive went ahead and accepted the contract. We are not quibbling with that, but why did the Scottish Government not intervene at that point, when the chief executive had called the legal department and said, “I don’t have the necessary skills to get under the skin of this.”?

It was not the chief executive.

Whomever it was, the SPPA got in touch with the Scottish Government’s legal department and said, “We don’t have the skills to get under the skin of this.” Why did you not intervene at that point? I have never run a Government, Ms Stafford, but if somebody told me, in relation to a huge contract involving a huge amount of public money, that they were not sure whether they had the staff who were qualified to ask the appropriate questions, I think that that would ring alarm bells and I would want to send someone in to sort it out.

The chief executive at the time had the skills and the track record. He had come from running a £15 billion pension operation.

So, despite the fact that the SPPA told you that it did not have the skills to ask the necessary questions, you were happy to let it go ahead and accept the contract.

The chief executive of the agency had the skills to reach the judgment, overall. He had done due diligence and he had received an assurance from the Scottish Government’s legal department that all the various assurances that he had sought specifically on Capita supported going ahead on the contract.

Yes, there was an additional element about asking an additional question. However, it is not that that the bid could not be probed at all; it was thoroughly probed.

What did your legal team say when the SPPA said, “We need help with this, because we don’t have the skills to ask the appropriate questions.”? How did the legal team respond?

I ask Gordon Wales to respond to that.

Can I go back—

If you answered that question, that would help me.

Yes, and I am looking at the correspondence. The legal department did not insist that the agency went back to Capita. It provided it with advice on additional questions that it could ask, but I go back to the point that the legal department confirmed that the responses that had been provided up to that point were reasonable. It provided advice on additional questions that the agency could ask if it wanted to. I want to be clear on that point.

Were the people in your legal department not concerned about the lack of clarity? Did they not think that they should flag up the issue further up the tree and maybe get intervention?

Again, they deemed the responses reasonable but provided additional advice on additional areas that could be probed if the agency wanted to do that.

What would happen now in the same scenario, permanent secretary?

As I have said, I do not think that the tender would have got to the stage that it has, for reasons that Colin Cook has described. The procurement process is completely different. In addition, there would be less pressure in terms of some of the conditions that the SPPA was experiencing at the time.

It is important to put it in a sidebar that the process was not happening in a vacuum. The chief executive had regular contact with his line manager and regular contact through the Fraser figure, who was attending meetings and received paperwork. The conversations and the tendering process that we are talking about were not taking place in a vacuum; they were part of regular contact between the chief executive of the agency and some of his staff, our staff and his line manager, and indeed, on occasion, ministers.

That would of course still happen now. We would have a very different set of testing and assurances, which Colin Cook has outlined. Equally, as part of that, there would be a much more in-depth and rigorous assessment of the nature of the contract, within the procurement guidelines that now exist, which are distinctly different from the ones that were in operation.

When was the contract awarded?

It was October 2015.

How long had the discussion of the contract been going on before that?

It would have been in the months prior to October 2015, during that year.

The bids were due in by a date in March. The date was adjusted slightly, but it was in March 2015, and then reviews took place throughout that time.

Exhibit 1 in the Auditor General’s report, which is on page 10, gives a timeline of the leadership and governance of the SPPA. It sounds as if you have read the Official Report of our previous evidence session when the committee probed the issue in quite a lot of detail. The conversations that we have just been talking about all took place when there was a serious hiatus at the top of the SPPA. You say that the contract bids were due in in March 2015. The permanent chief executive, who had been in the post for 13 years, left the following month, in April 2015. There was then a three-month gap with an interim chief executive, who left in June 2015. A permanent chief executive was appointed in July 2015, and then that chief executive was pulled into the Scottish Government—I understand to work on another project—until September 2016, and the agency was left with a senior responsible officer.

Given the points that we have just covered about the questions that needed to be asked about the contract and the big question marks over it, did the lack of leadership at the top of the agency have anything to do with those issues?

I need to correct some of those points. An accountable officer and chief executive officer has been in place at all times for the SPPA. Ross Paterson took up the post in May 2015. He was in place for two years and saw through the concluding stages of the evaluation and tendering. He saw through the contract award in October 2015. He was also wrestling with Capita to try to ensure that the project stayed on track and was able to deliver. Because Capita failed to do that, Mr Paterson was also responsible for having to extend the existing contract to ensure that there was provision and that customer care and customer focus were maintained.

At one point, he went for about three months to work partly for the Scottish Government, but he retained his responsibility as chief executive of the agency and as accountable officer. That was agreed with the chair of the management advisory board and the cabinet secretary at the time. He maintained responsibility for the project throughout those three months or 12 weeks and he attended all the project board meetings during that time. He was the key player overseeing the process and in all the contact with Capita during those three months. Therefore, I contest the assertion that there was a lack of continuity.

There were some personal circumstances, which we had to adjust to and address but, in all cases, we were trying to ensure that we had a balance between business continuity and the duty of care as an employer. We do not need to go into what those circumstances were.

Indeed not.

When you had such an important contract with considerable public expense and such a big on-going project, why would you give somebody a second job in Government? Why did you not leave them there and let them properly manage what was going on?

I spoke earlier about the balance between business continuity and the importance of delivery, on the one hand, and personal circumstances and our duty of care as an employer on the other. Taking into account all those circumstances, we decided that the best thing to do was to get a working arrangement that de-risked the tension between those two sides. That meant that Mr Paterson was working for the Government for a certain period of time on a particular project—which Alyson Stafford can describe, and where he had valuable and important experience—but continued to be responsible for overseeing and driving the delivery of the IT project.

I could go into a few more details about the personal circumstances—

I do not want any personal circumstances—

I did not think that you would.

I did not even think it was necessary to mention anyone’s name here.

It was important, because that was one of the reasons why we slightly changed the configuration of responsibility at the agency at the time. The deputy was in charge of day-to-day activity in the agency, but Ross Paterson retained accountable officer responsibilities, chief executive responsibilities and, indeed, responsibility for this project. There was no doubt about that.

Permanent secretary, would you not agree that it seems odd that, if you have just signed a contract and if there are issues about awarding it—as is clearly set out in paragraph 15 of the Auditor General’s report—you would give somebody a second job on top of overseeing what was a very important job at the same time?

We do not want to get into personal circumstances—

I am not asking you to get into personal circumstances.

No—but you are doing that by the question that you are asking, I am afraid. I am not going to go into what they are, but we had to ensure that, in order to maintain the continuity of business, in order for the chief executive to be able to continue to devote his important time to leading and indeed project managing this important piece of IT development, and in order to facilitate his doing that, to have him working a little closer to his home and supporting an important project—which Alyson Stafford can describe—was a good business solution. That approach combined business continuity—because he was still in charge of the project—with duty of care as an employer.

You have left us in an awkward position. I would be the last person who would want to dive into anyone’s personal circumstances, but it seems an odd way to deal with duty of care to give somebody a second job, no matter where that job may exist.

It was not a second job; it was another part of work that supplemented what he was doing. He was not in charge of the day-to-day running of the agency any more, but he was in charge of this project, and he had oversight as AO, as well as chief executive responsibilities. I do not want that to be confused.

Why would you want to change the person at the top of the organisation in the middle of such a critical project?

We sometimes have to take decisions about a duty-of-care responsibility, supporting individuals, the location of their work and their working pattern where we can ensure that that does not put projects at risk—which this decision did not—but in a way that enables us to be responsible employers.

I will move on to the subject of governance. At a previous meeting I asked some questions about the SPPA’s audit committee. In summary, I had some difficulty finding out from the website who the audit committee were, and I found the committee’s minutes to be subsumed into the minutes of the management committee, I think it was. Only recent minutes were on the website, and we had to email to get previous ones. We did that, but I do not think that we have received them.

When you receive a section 23 report such as the one that we are discussing, do you ask yourself—in this case—what the audit committee was doing and whether it is fit for purpose?

Of course, we reflect very seriously on reports such as this. You will have been able to tell from the evidence sessions that you have conducted that we think very carefully about the learning around all this.

The audit committee met, and I believe that its meeting was attended by our Fraser figure. We also had officials regularly attending the agency’s management board, audit committee and project board. We had a Government responsibility, through the Fraser figure, to be in touch with the governance and assurance of the SPPA. If we feel, as a result of the Auditor General’s report, that those need to be tightened, that will be one of the things that the SPPA will be responding to in due course.

I do not know whether the audit committee met and reported, because I have not been able to see the earlier minutes. I do not know whether you have been able to look at those yourself.

I would have to ask colleagues here whether they have, although much of the information is archived, as you might understand.

However, one thing that I will say, which may be helpful to you, is that we have our own Scottish Government assurance and audit committee meeting next week, on 10 December. The section 23 report, the SPPA’s response to it and our Government response, including some of the issues that you have been asking me about, will be considered. We will then take further advice and deliberate about what Scottish Government circumstances we need to change in our relationship with arm’s-length bodies. I know that the SPPA has already shared with you some of the changes that it has made as a result of its deliberations on the report.

I do not want to broaden this to other audit committees, but I know that the Auditor General had comments about your Government audit committee.

I am concerned that issues have been raised, across a number of reports, about what the SPPA audit committee was doing and I would really like to know that you have looked at this section 23 report and that you have satisfied yourself that that audit committee is now at least fit for purpose.

Yes. I know that Alyson will want to talk about that. We have asked the SPPA to ensure that, in its comprehensive response to the Audit Scotland report, it reflects on the governance and assurance arrangements that it had in place then and those it has in place now. We are not complacent.

But if the organisation itself has had an issue and a problem, can you just rely on it to say that its audit committee is functioning properly?

It will be for the organisation in the first instance to look, as it is doing, at the nature of the report and what it says about its audit and risk practices. Alyson may want to say something about that in a while. It is also for us to ensure that we are playing our role in relation to an arm’s-length body with an accountable officer and its own board, and to ensure that there is no room for things to fall between those two stools, on audit and risk capacity. Alyson, do you want to say something about that?

Certainly. The audit and risk committee at the Scottish Public Pensions Agency was in place during this time. It met according to the schedule one would expect it to meet.

This is going back to 2014?

Yes. It did meet. It was populated, as you would expect, with non-executive directors and officials were in attendance. Internal audit services and Audit Scotland would also have been in attendance at those meetings throughout that time. The non-executive directors who were part of that included people with large-scale, public sector financial experience; they would also have brought in private sector experience. That continues up to today. Those people have experience as accountable officers, so they would understand that. There was also someone who had only recently stood down after completing their term of office, who also had specific private sector pension industry experience, from Edinburgh. That was helpful because, as well as this project that we are talking about, there were another seven projects going on, implementing not only changes that the Scottish ministers wished to see—as the permanent secretary has already said, bringing in between 20,000 and 30,000 firefighter and police pensions—but the Hutton reforms from 2011, which was a UK Government requirement.

You have the benefit of having seen the minutes: can you tell us whether the audit committee flagged up the issues, or did the issues fly past it?

The audit committee was briefed about the projects. There was an overall transformation programme that was programmed to run—

Did the audit committee raise issues that somebody, presumably, ignored?

The audit committee scrutinised each stage and was equally testing about how Capita was to deliver on the contract. The main issues were not really around after the contract had been let, but about the lack of contract delivery. To come back to Mr Coffey’s earlier question, Colin was not able to comment specifically on the software development, but we know from some other material that—

It may be helpful if I directly address Mr Coffey’s comment. A 2016 independent first review of the project states:

“Capita are using a waterfall approach to development and apply recognised international standards to the development process”.

That is an independent report.

I realise that we need to address your specific question. The audit and risk committee’s issues were to do with Capita’s non-delivery in respect of what was required at particular points in time. In the first stages of the contract, Capita delivered on various elements but, at the end of July or early August 2016, a particular element of the software supply was applied to user acceptance testing in the agency. The agency staff are considered to be part of the cost of the project, because they tested the software. They had the experience and the knowledge, and they were used to using previous systems and the reforms that were being brought in. At that point, the success rate of the calculations that the software generated was not satisfactory. The audit and risk committee was therefore briefed and aware of the issues. The project board then intensified its activities in holding Capita to account through the process.

Did the audit and risk committee raise issues such as the continuity of the management at the top of the agency?

Not to my knowledge, because there was a managed handover. There was a short period between the chief executive standing down at the end of March 2015 and the interim management. There was then a long handover between the interim chief executive and the substantive chief executive. The audit and risk committee and those members who also had a link into the management advisory board were sighted on that and recognised the actions that were being taken.

Perhaps there is a takeaway from this. It is hard to argue with you, because I have not been able to see the reports. Will they be made available? I do not mean just historical reports; I mean on-going reports.

I am sure that we can arrange to share the historical ones, but in terms of—

I mean will they be on the SPPA’s website? I do not necessarily want them myself.

I am sure that the SPPA would want to dwell on that and ensure that it has the right kind of information on its website.

The convener asked about the short period of time in which the chief executive of the SPPA worked on a financial project as well as having oversight of the project. That was agreed by the management advisory board. It was not as if that work was carried out separately. The management advisory board was aware of that.

I have a final, more general question. If you get a section 23 report, do you ask yourself what the body’s audit committee has been doing and whether it is still operating functionally?

I can speak only on behalf of my own audit function; Alyson Stafford may want to talk about arm’s-length bodies.

You can talk about the bigger picture.

That function is, of course, a very important part of the governance and assurance of any organisation. You referred to Audit Scotland’s comments on the Scottish Government’s audit and assurance committee. We have taken those comments extremely seriously.

I do not want to go into them specifically.

Absolutely. The function is very important. It is an integral part of the Government’s assurance of any organisation, and particularly the assurance of a Government body or a body that is at arm’s-length to the Government.

Mr Bowman is referring to the audit and risk committee during a particular period. As I mentioned, Audit Scotland has been present at that committee. The Audit Scotland reports on the SPPA’s accounts for the 2014-15 and 2015-16 financial years made no reference to the project. The Scottish Public Pensions Agency made a tremendous effort, and subject matter experts were brought in. The chief executive at the time approached me to talk about bringing in people who would help with continued diligence on the delivery of the project and the change process. The project was one of eight. At the time, major things were happening to pensions in Scotland, which the agency led on. Those eight projects included software elements. Prior to that, the Scottish Public Pensions Agency delivered successful software changes, and it has done so subsequently.

The real concern was about the supplier’s non-delivery. To give you a sense of the diligence that was applied in managing the contract, the project board met at least monthly and the frequency intensified as soon as there were issues. Further, the SPPA approached the Government to ask to be part of the pilot of the technical assurance framework that has been designed and is in place. Therefore, the chief information officer became a member of the project board. There was a huge amount of effort and scrutiny around the supplier to make sure that it delivered.

When the first failure occurred, a tremendous amount of work was done, not to look at the sequencing and critical path of the project—one thing after another—but to work through which elements could be built coterminously. We are not critical of that. It was about building them side by side, so as to make up the time.

The key thing was that there was a hard deadline for the project. March 2017 was an absolute cut-off date, for one reason alone. The project integrated administration, payroll and web-based services and, of the three existing suppliers that were to be replaced by one supplier, two were willing to extend the contract beyond March 2017, but the other refused to do so, which created that hard deadline.

Subject matter experts were brought in, which enabled us to redesign the project as it went along. It was Capita that—one series after another—failed to deliver.

We appreciate the detail, but will you try to be briefer?

Yes, of course. Those are some of the key points. That was the capacity that was there to support the project and keep it as on track as possible.

The only other point to draw out about the effective management of the contract is that, according to the contract terms, Capita could and should have earned £2.7 million if it had delivered the project. That is the contract price that would have been paid during the period that we are talking about. In the end, the SPPA paid less than £700,000, which shows that it did not pay for anything that it did not get. The contract was rigorously managed. The sad thing about it was the lack of delivery.

We had an evidence session with the SPPA on 26 September, which covered some of the changes to sponsors during the life of the project. Gordon Wales advised the committee that the sponsor arrangements encompassed line management of the chief executive and the public body’s Fraser figure. At different times during the project, those roles were combined and carried out by one Scottish Government official; at other times, they were split between two officials. A Scottish Government sponsor official is also a non-executive director of the SPPA’s management advisory board.

I am looking at the timelines for the to-ing and fro-ing of the sponsors. That surely must have contributed to the chaotic management of the project. What were those guys doing during that time? The sponsors must have been aware of the problems, so what did they do?

On your first point, Gordon Wales now has the roles and responsibilities of line manager and Fraser figure. That is slightly different from the sponsor process, but I understand the point that you are making.

The Fraser figure is a very particular role, which embodies the responsibilities that are delegated from a minister to a director general. The director general decides whether to take on those roles and responsibilities themselves or to give them to a Fraser figure. I might be wrong—I am willing to be challenged on this—but I do not recall another occasion on which we have combined a Fraser figure with line management responsibility. I was not permanent secretary for a good chunk of the project, but as far as I am aware, Gordon Wales was the first person to be the Fraser figure and hold the line management responsibility.

According to my information, the roles were also combined from April to June 2015.

I am not aware of that being the case. Your point was about the quality and nature of the Fraser figure role. Eleanor Emberson—later, Eleanor Ryan—and Alistair Brown, who acted as Fraser figures from 2014 up to Gordon Wales’s time, were two of our most experienced finance officials. They worked for Alyson Stafford and were colleagues of Colin Cook, and they were both very experienced not just in relation to the Fraser figure responsibilities but in the implementation of major IT projects. They would have been very clear about their roles and functions—Gordon Wales can talk a bit more about that—and about their responsibilities for attending board meetings and seeing the management advisory board papers, which they did. Alongside that, they had line management responsibility—with Alyson Stafford, for the most part. Those two functions were a very important part of our being kept abreast of what was going on. Ministers were also kept abreast of the process.

The Fraser figures would have been familiar with the rigorous process that Alyson Stafford described and with the management of the Capita relationship, particularly by Ross Paterson, who managed it at the most crucial time. The chief information officer was also on the board, and she and her colleagues were part of the independent testing at three different times during the project’s life.

I presume that whoever was discharging that function for the Scottish Government was well aware of the problems with the project. How did they discharge their duties? How did they assess the risks to success? How did they report that up the line? I feel that the problems should have been flagged up earlier.

You are right that the Fraser figure’s role is to support but also to challenge on behalf of the Scottish Government. The Fraser figure would have a relationship with the minister and with line managers, in terms of reporting back.

The accountable officer had the role of and responsibility for executing the project and keeping the Government appraised, which he did regularly. It was not that we were not aware of what was going on; we were part of the solution and part of the testing and probing of Capita. We looked at what it could deliver, particularly at the three junction points that are mentioned in the independent IT review. You will have read about the amber, the amber-red and the red categories in the report and the accompanying documents.

Does Alyson Stafford want to say anything about the Fraser figure information that was relayed to her?

Yes, certainly. Through regular line management contact and through the Fraser figure feedback from the various project and management advisory boards, there was a clear awareness that what Capita had been contracted for was not being supplied by the required deadlines.

The first indication of that lack of supply came at the end of July and into early August in 2016. A further extraordinary project board session was set up in the middle of August. I personally went to that session to meet the senior representative, who was Capita’s signatory to the contract, to stress that we expected and required the contract to be delivered. I insisted that that needed to happen. I also attended the session to see the replanning and reprofiling that I described, with separate blocks of activity identified, which would be done coterminously, not in sequence.

I wanted the Capita software development team to have a more regular presence on site, because a lot of its work was being done remotely. Space was made available in order to get people from Capita on site, so that they would be there and could test anything that they wished to by way of interpretation of regulations and specification. As I say—

I am sorry, but we are running short of time. Would you like to add anything else?

There was oversight and there was knowledge. Ministers were briefed at key points when various things were happening. Equally, when there was the particular change in October—a recognition that Capita was going to fail to meet its March deadline—specific actions were taken.

Thank you.

Just to add to that, the work that was done at the specific points relating to the three levels of assurance, and the scrutiny that was done by the chief information officer, were really important. That was an independent means of holding Capita to account not only during the contract but, sadly, subsequently, when it had to be closed.

We can give some information in writing on that, if that would be helpful.

That would be very helpful.

I thank the witnesses for their evidence this morning.