Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national...

88
NeI Survey Report - 2015 Project Directors Group (PDG) Report of 2015 UK National e-Infrastructure Survey of HEIs and Research Institutes Martin Hamilton (Jisc/NeI PDG - editor) Clare Jenner (UCL/DiRAC/NeI PDG) Jacky Pallas (UCL/eMedLab/Farr/NeI PDG) Alan Real (Leeds/N8/HPC-SIG) Andrew Richards (Oxford) Jeremy Yates (UCL/DiRAC/SKA/NeI PDG) UK National e-Infrastructure Survey coordinated for Project Directors Group by Jisc

Transcript of Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national...

Page 1: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

Reportof2015UKNationale-InfrastructureSurveyofHEIsandResearchInstitutes

MartinHamilton(Jisc/NeIPDG-editor)ClareJenner(UCL/DiRAC/NeIPDG)JackyPallas(UCL/eMedLab/Farr/NeIPDG)AlanReal(Leeds/N8/HPC-SIG)AndrewRichards(Oxford)JeremyYates(UCL/DiRAC/SKA/NeIPDG)

UKNationale-InfrastructureSurveycoordinatedforProjectDirectorsGroupbyJisc

Page 2: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

ExecutiveSummary 2

Contents

ExecutiveSummary............................................................................................................................................3

Progressreportonrecommendationsof2014NeIsurvey..................................................................................6

E-Infrastructurecapitalinvestmentsandmajorinitiatives................................................................................11

TheStatusoftheRegionalandNationalHPCProjects.............................................................................................12

Regionalconsortia.....................................................................................................................................................12

TheNationalHPCProjects.........................................................................................................................................13

BootstrappingtheAlanTuringInstitute....................................................................................................................13

WhoaretheUK’snationale-Infrastructureproviders?.....................................................................................14

SustainablefundingfortheNeIEcosystem........................................................................................................16

TheHEIProblem........................................................................................................................................................16

NonHEIProviders.....................................................................................................................................................17

E-Infrastructureasasharedservice...................................................................................................................18

Industry............................................................................................................................................................19

Examplesofindustrycollaborations.........................................................................................................................20

AppendixA-Listofsurveyorsandacknowledgements.....................................................................................22

AppendixB-Whowillreceivethissurvey.........................................................................................................22

AppendixC-Listofrespondents.......................................................................................................................22

AppendixD–Servicemanagement:Thesurveyquestions................................................................................24

AppendixE–Servicemanagement:Summaryofthesurveydata......................................................................26

AppendixF–Servicemanagement:Fullbreakdownofsurveydata..................................................................28

AppendixG–Hardware:Thesurveyquestions.................................................................................................54

AppendixH–Hardware:Summaryofthesurveydata......................................................................................56

AppendixI–Hardware:Fullbreakdownofsurveydata....................................................................................58

AppendixJ–Hardware:Summaryofthesurveydata........................................................................................78

Page 3: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

ExecutiveSummary 3

ExecutiveSummaryThisisthedraftreportofthethirdannualUKNationale-Infrastructure(NeI)Survey.ThisisintendedtoinformthemanagementanddevelopmentoftheUK’sNationale-Infrastructureforresearchandinnovation.TheNeIiscomprisedofpubliclyfundedfacilitiesandexpertisethataremadeavailabletoacademicresearchersandforindustrialandthirdsectorengagements.ThereisafulltableoffacilitiesavailableinAppendixJ,includingArcher,theUK’snationalservice,andsystemsprovidedbytheSTFCHartreeCentre.

ThissurveyreportsummarisesthecurrentstateoftheUKNeI,reportingonrecentcapitalinvestments,andintegrationactivitiesdesignedtomaketheNeImoreeffectiveandusercentric.Wemakeanumberofrecommendationsbasedonthefindingsfromthissurvey,whichwillhelptoplacetheNeIonamoresustainablefootingandensurethattheNationale-InfrastructureunderpinsUKresearchandinnovationthatisgloballyleading.

Recommendations:

1. Trainingmaterialsandcoursesshouldbeeasilyavailable.Aservicetohostmaterialsandadvertisecoursesandtoassigncoursestoparticularlevelsofskillandknowledgewouldbeveryhelpfulinmakingsureusersgetappropriatetrainingandareabletoprogresseasilyfromleveltolevel.HEIsandResearchDomainsshouldbeencouragedtosharewhatmaterialsandresourcestheycanwitheachother.Traininghastobeprovidedfortrainers.

2. ATrainingFramework,possiblyonanon-lineMarketplace,needstobecreatedtoallowouruserbasetofindoutandaccesstherighttrainingneededfortheirproject.

3. RCUKshouldasamatterofurgencyprepareabusinessplanforcapitalandoperationalresourcestosupportHPCprovisionintheNeI,indicatingtowhatextentthereductionininternationalcompetitivenessandlossofproductivityduringtheperiod2016-18canbemitigated.ThiswillbenecessaryinordertorealisetheambitionsofinitiativessuchastheTuringInstitute.

4. ThenewlyestablishedRCUKCloudWorkingGroupshoulddevelopastrategytoestablishthescopeandpotentialofCloudservicesforresearchandinnovation.ThisshouldidentifywhichpartsoftheNeIcouldusefullyadopthybridCloudtechnologies,andhowCloudtechnologiescouldassisttheNeIinsharingresourcesbetweenitsdifferentprojects.

5. ToleverageUKinvestmentsinEUandinternationale-InfrastructuressuchastheEuropeanScienceCloud,EGIandEUDAT,RCUKshouldreportonthecurrentlevelofUKparticipationintheseprojectsanddeviseastrategytoensuretheUKbothmaximisesuse,andinfluencesthedesignandmanagement,oftheseinfrastructures.

Page 4: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

ExecutiveSummary 4

6. HPC-SIGhasakeyroletoplayinfacilitatingNeIproviders’self-organisationintopartnerships/consortiatobuildcriticalmassofknowledge,achieveoperatingefficienciesandmaximiseproductivity,creatingeconomiesofscalewithoutcompromisingdiversity.

7. AneconomicsustainabilitymodelofthecurrentNeIshouldbeconstructedasamatterofurgencyandasustainabilitystrategydevelopedbythemainparticipantsoftheNeI.Workshopswillbeheldtodevelopthismodelandtodeveloppartnershipsinserviceprovision.ThisisnecessarytoassistwithplanningandtoallowcomparisonofservicecostswithPublicCloudproviders.

8. E-Infrastructureprovidersneedtobegivenconfidencethatthereisalongtermplaniftheyaretoreleasefundsthatwillunderwritestaffrolesbeyondthelifetimeofcurrentprojects,andtodevelopacareerstructureforstaffworkingin“ResearchOperations”(ResOps).

9. RCUKshouldproposecommonresearchandinnovationrelatedmetricsforallNeIproviderstoensureastandardofreportingtoallowtheeffectiveness,efficiencyanddiversityofNeIserviceprovisiontobeassessed.

10. ServiceProvidersshouldbeencouragedtoformpartnershipsandpoolresourcestodelivereconomiesofscaleinprovidingafullrangeofservicesacrossacademic,industryandthirdsectororganisations.

11. Increasetrainingandupdateservicemanagementprotocolsappropriately,sothatthoseusingandmanagingNeIservicesareawareofthesecurityrequirementsoftheirprojectsandtheirsystemsrespectively.Thisshouldbedoneinaproportionatewaythatreflectsdatabeingprocessedanduses/usersofthesystems.

12. ExplorethepotentialofusingJisc’sVATcostsharinggrouptoreducethefrictionofsharingfacilitiessuchase-Infrastructure,andresearchequipmentmoregenerally.

13. TheELCshouldorganisemeetingsbetweenregionalconsortia/partnershipsandNationalProjectswithLEPsandinitiativessuchastheNorthernPowerhouseandMidlandsEnginetoensurethatlocale-Infrastructureneedsareassessed.

14. IUKshouldworkwithregionalconsortia/partnershipsandNationalProjectstodeveloptheCatapultsandworkeffectivelywiththeresearchcommunityandgovernment.ExamplesincludethePrecisionMedicineCatapultwhichwillhaveitsHQinCambridgeandtheMedicalTechnologyCatapultwhichbelocatedclosetotheHartreeCentre.

15. IndustryshouldhaveimprovedvisibilityofNeIresources,e.g.viaaccesstodashboardsandportalsdevelopedbytheNeIforitsownuse.

Page 5: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

ExecutiveSummary 5

16. Regionalandnationalfacilitiesshouldhaveanindustrialengagementstrategyincludingmetricsthatpermittheeffectivenessofthestrategytobequantified.

17. LeverageexistingandnewRCUK-fundedbusinessdevelopmentstafftosupporte-Infrastructurerelatedelementsofindustryengagement,workingwithregionalcentresandlocalHEIs.

18. EncourageuseofstrategicfundingsuchasImpactAccelerationAccountstopump-primebusinessdevelopmentactivitieswithinHEIsspecificallyforcomputeanddataresources.

19. ThenewAAAIapplicationSAFE+AssentandthesecureaccessandsecuredatatransportinfrastructureSafeShareshouldbetestedandproductisedinreadinesstoberolledouttotheNeI.Asustainabilitymodelisalsoneededforboththeseservices.

20. End-to-endnetworkconnectivitydiagnosticsshouldbegintobeundertakenbetweenNeIprovidersandusers’localresearchorganisationstoidentifynetworkbottlenecksandprovideresearchorganisationswithadiagnosisreport.

21. CommoncoreapplicationsfortheUKDatae-Infrastructureinordertomoveandselectdatafromdistributeddatasetsshouldbedevelopedandtested.

22. AnRCUKmetadatastrategytobedevelopedtoensurethatmetadataisgeneratedatthepointofdatacreation,andthatstandardmetadataqueryandanalysistoolsareprovidedtoUKresearchers.

Page 6: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

Progressreportonrecommendationsof2014NeIsurvey

1.TheconnectionsofNeIServiceproviderstotheSJ6backbonebeevaluatedandifnecessaryupgradedorseparatelinksbeprovided.

2.Internalinvestmentbyinstitutionsisrequiredtoensurethatinternalcampusnetworksremainfitforpurpose.ThisneedstobecommunicatedtoHEIsviatheirPro-VCsforResearch.

JischaverecruitedateamledbyTimChowntodeliveranend-to-endnetworkconnectivitydiagnosisservice.FromtheAutumnof2015thisteamwillworkwithNeIProjectsandHEIstotesttheefficacyoflinksfromlocalinstitutions’servicestotheSJ6network.Thesetestsandasuggestedinvestmentstrategywillbemadetotheparticipatinginstitution.Thisshouldeasedatacongestionwithinthenetworkinfrastructureoperatedbytheinstitutionandincreaseresearchproductivity.

3.AlongtermcapitalplanisrequiredtoensurethefutureproductivityoftheNeI.Thisshouldbeco-ordinatedandcarryonmomentumestablishedtocreateaholistice-Infrastructureeco-systemfortheUK.Beingabletoplanwillgreatlyincreasetheefficacyandefficiencyofoursystemsandreleaseresourcesforaddedvalueactivitiessuchassoftwareengineering.Greatercoordinationhasthepotentialtoestablishdeeperandmorevaluablepartnershipswiththevendorcommunity.

TheUKNeICommunityproducedfourkeyreportstoaddresshowwecreateanintegratedholistice-Infrastructure:

• TheRCUKE-InfrastructureRoadmap,producedbytheRCUKNeIGroup,whichputforwardacoherentstrategytointegrateanddeveloptheUKNationalE-InfrastructuresothatitcandriveforwardthecontinueddevelopmentofagloballycompetitiveresearchbasewithintheUK.Seehttp://www.rcuk.ac.uk/RCUK-prod/assets/documents/documents/RoadmapforELC.pdf

• TheRCUKDataforDiscoveryWorkshopReport,producedbytheRCUKNeIGroup,thatproducedrecommendationsfortheformationofaUKNationalDataforDiscoveryInfrastructure.Seehttp://www.rcuk.ac.uk/RCUK-prod/assets/documents/documents/RCUK%20DataforDiscoveryWorkshopReport.pdf

• CloudComputingforResearchandInnovation,producedbytheNeIProjectDirectorsGroup,whichproducedastrategyformakeuseofcloudtechnologiesandcloudproviderstoenhancee-infrastructureoutcomesin

Page 7: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Progressreportonrecommendationsof2014NeIsurvey 7

theUK.Seehttps://www.scribd.com/doc/273829152/Cloud-Computing-for-Research-and-Innovation

• ImaginingtheUKNationalDatae-Infrastructure,producedbytheNeIProjectDirectorsGroup,whichidentifiedthecoretechnologiesrequiredtopowerafutureUKNationalDataforDiscoveryInfrastructureSeehttps://www.scribd.com/doc/260531862/Imagining-the-UK-National-Data-Infrastructure-

Recommendations

TheRCUKNeIGroup(Chair:Morrell)continuestoco-ordinatee-InfrastructurestrategyamongsttheResearchCouncils,InnovateUK,JiscandtheMetOffice.

InadditiontheRCUKNeIGrouphasfunded(£40k)theNeIProjectDirectorsGroup(Chair:Yates)sothatitcanbetterco-ordinatee-infrastructureintegrationactivities.ThishasallowedthePDGtoorganisemeetingsandworkshopson

• Authentication,AuthorisationandAllocationInfrastructure(AAAI)• CloudTechnologyandCloudProvision• CoreTechnologiesforaNationalDataInfrastructure• DisruptiveHardwareTechnologyanditseffectsonSoftwareneedsanddevelopment

RCUKandPDGmembersmadeinputsinto2016-2021scienceandcapitalplanhttps://www.gov.uk/government/publications/our-plan-for-growth-science-and-innovation

4.JisciswellplacedtocontinuecoordinatingeffortsintheAAAIarea,buildinguponexistingworkwhereverpossible.

JiscannouncedthenewAuthenticationInfrastructure,formerlyknownasProjectMoonshot(seehttps://www.jisc.ac.uk/assent)tomanageaccesstobothwebandnonwebservices.ThiswillformthebasisofasinglesignonservicefortheNeI.

5.DiRAC,GridPPandJiscworktogethertoproduceaprototypethatallowstheseNeIProjectstoshareresources.Thisteststhesingle-sign-oncapabilityoftheproposedcommonAAAIinfrastructure.

Jisc,eMedLab,DiRAC,GridPP,TheUniversityofOxford,TheN8RegionalHPCServiceandtheEdinburghParallelComputingCentreareparticipatingintheSAFE+AssentProjecttoproduceaprototypeAAAIservicethatcanprovidesinglesignonservicesfortheNeI.ThiswillincludeaprojectaroundsecureaccessandfederatingwithprojectsthatareunabletouseAssentsuchasGridPPandElixir.

6.eMedLab,FarrInstitute,ARDC,EBI/ELIXIRandJiscworktogethertoproduceasecureAAAIinfrastructurethatprotectsthesecurityofthesensitivepeopledata.ThisteststhedatasecurityaspectoftheproposedcommonAAAIinfrastructure.

Page 8: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Progressreportonrecommendationsof2014NeIsurvey 8

JischavefundedaprojectSafeShare,£960k,whichprovidesbotha2factorAuthenticationinfrastructureandsecuredatatransport(seehttps://community.jisc.ac.uk/groups/safe-share-project).ThisdoesnotincludetheLifeSciencesprojectELIXIRwhohavetheirownsystem,developedinFinland,butthiswillbeableforfederatewithSafeShare/Assent.

TheSafeShareprojecthasanobviousroleinprovidingsecureaccessdistributeddataanddatatransferavailabletoothersectorsintheeconomysuchasHealthandAdvancedManufacturing.

7.JASMIN,SangerInstitute,GridPP,SKAandDiRACshouldplantoworktogethertoexplorethepracticalitiesofthisapproachandshowthatresourceswithinResearchDomainscanbeconfiguredintoeffectiveandefficientprivatecloudsthatallowresearcherstoruntheirworkflowseasilyonadomainprivatecloudoronresourcesinanotherpartoftheNeI.

GridPP,STFCScientificComputingDivision,SKAandDiRAChaveformedanumbrellaprojectUK-T0todevelopthisapproach.ThisbuildsontheprogressalreadymadebyprojectssuchastheCERNCMSexperiment,theMRCMedicalInformaticsCLIMBandeMedLabprojects,theEBI’sEmbassyCloudandJASMIN2.

SKAhavebegunaprojectwiththeUKsoftwarecompanyCanonical(http://www.canonical.com/)todeveloptheOpenStackCloudoperatingsystemforuseindataintensiveprojectsandapplications;thusmakingOpenStackmoreattunedtotheneedsofUKdataintensiveprojects.

8.Jisctotakethiswork(publiccloud)forward,incollaborationwithmajore-Infrastructureserviceusersandpubliccloudproviders.Howeverthecostsandbenefitsofsuchaccesswillhavetobecarefullymeasuredanditisunlikelytobeasolutionformanyofourproblemsetsinthenext2-3years.PublicCloudProvidersandNeIprovidersshouldexchangeinformationconcerningtheproblemtypesandsizesintheNeI,sothatexpectationsarenotundulyraisedandthatmethodologiesarebuiltupthatalloweffectiveandeconomicuseofPublicClouds.

JischavesurveyedtheCIOsoftheUK’sHEISectortoassessCloudusageingeneralintheHeISectorandfoundthattherewerearangeofissuespreventingmorewidespreadadoptionofcloudincludinglegalandregulatoryaspectsandlackofclarityoverpricingandcostingfor“bogstandard”enterpriseITapplications.See:

https://www.jisc.ac.uk/news/uk-education-divided-in-its-adoption-of-the-cloud-14-jul-2015

JisccontinuetoworkwithPublicCloudprovidersonaccessportalsandegressontotheSJ6backbone.JiscareinvestigatingtheofferingofabrokeringservicetoallowmoreeconomicuseofPublicCloudservices.

ThePDGproducedareportCloudComputingforResearchandInnovation.

RCUKNeIGrouphaveestablishedaCloudWorkingGroup(Chair:Kershaw,RAL),todrivethedevelopmentofprivatehybridcloudsintheNeIandimproveaccesstoprivateclouds

Page 9: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Progressreportonrecommendationsof2014NeIsurvey 9

AnOpenStack(theleadingopensourceCloudTechnology)UserGrouphasbeenformed,supportedbytheUKSMEOCF,toacceleratetheuptakeofCloudtechnologiesbyUKNeIproviders.

9.Thee-InfrastructureLeadershipCouncilisencouragedtoconsiderpotentialapproachessuchasgreaterregionalcollaborationsupportedbyanelementofmatchedfundingfortheHEIs’e-Infrastructureinvestments.ThisshouldbehighlightedtoPro-VCsforResearch.

WewelcometherefreshedcommitmentfortheE-InfrastructureLeadershipCouncilunderthenewadministration.TheirroleinprovidingstrategicadvicetotheNeIcommunityisvaluableparticularlyaswegrowourengagementwithindustry.

ThissurveyreportmakesanumberofrecommendationsonhowtheNeIcanbettersupportIndustryandspecialistareassuchasHealth.

10.ThereisnowtheopportunitytoformallyrecogniseanddefinethecontributionoftheNational“Centres”totheNe-Iandtomakesuretheyareadequatelyfundedtocarryouttheirindividualmissions.

Thisstillneedstobedone.

11.Fundersshouldmakeapplicantsawarethatitispermissibletoapplyforresearchsoftwareengineertimeongrants,andthatitisalsoappropriatetoclasstheseasresearchstaffongrantswheretheworktobecarriedoutinvolvesasignificantresearchanddevelopmentaspect.

Thishasbeendone,althoughitwilltaketimetoworkthroughthesystemasgrantpanelsneedtounderstandthisworkimprovestheproductivityofresearchprojects.

12.Theworktoraisetheprofileoftheresearchsoftwareengineershouldcontinue-inparticulartherole,valueandpotentialcareerpathsshouldbehighlightedintosubmissionstotheELCandHEIs.ThevalueofdedicateddevelopersupportatHEIsshouldalsobehighlightedtoPro-VCsforresearchandDirectorsofResearch,aswellastosuccessfulfundingmodels.

TheEPSRCcallforResearchSoftwareEngineeringFellowshipswasissuedinMay2015,providingsupportofupto£3.7MforResearchSoftwareEngineers.,andanetworkledbytheSoftwareSustainabilityInstitutewhichisopentoallResearchSoftwareEngineers.

Seehttps://www.epsrc.ac.uk/funding/calls/rsefellowships/

13.Trainingmaterialsandcoursesshouldbeeasilyavailable.Aservicetohostmaterialsandadvertisecoursesandtoassigncoursestoparticularlevelsofskillandknowledgewouldbeveryhelpfulinmakingsureusersgetappropriatetrainingandareabletoprogresseasilyfromleveltolevel.HEIsandResearchDomainsshouldbeencouragedtosharewhatmaterialsandresourcestheycanwitheachother.Traininghastobeprovidedfortrainers.

Page 10: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Progressreportonrecommendationsof2014NeIsurvey 10

14.ATrainingFramework,possiblyonanon-lineMarketplace,needstobecreatedtoallowouruserbasetofindoutandaccesstherighttrainingneededfortheirproject.

TheArcherNationalServicehasmadeplacesonitscoursesavailabletothosenotfundedbyEPSRCandNERC,providedresearchersfundedbythoseResearchCouncilsalsoattendthosecourses.ThesecoursesareheldnowheldatmanysitesaroundtheUK.

TheArcherNationalServicehasproduceditsownDrivingTesttocheckandimprovethebasicITskillsofitsnewusers.BothDiRACandArchernowdemandsuchtestsaretaken.

TheSoftwareSustainabilityInstitutecontainstofocusonimprovingtheskillsofthoseenteringtheNeIviatheHEIsector.Theseincludetrain-the-trainertrainingsessions,softwarecarpentryworkshopsandtheprovisionofteachingmaterials.

DiRACandArcherhaveproducedamanycoreprogrammingcourseandaprogrammingcoursefortheIntelXeonPhi,whichwillbemadeavailabletotheNeI.BoththeSKAandSES5haveproducedmanycoreGPUprogrammingcourses.

DiRACandArcherhaveproducedacodeoptimisationcoursethatwillbemadeavailabletotherestoftheNeI.

TheHartreeCentrecontinuestoofferits3weekSummerSchooltotheacademiccommunityandindustry,aswellasafullprogrammeoftrainingeventsaimedatindustry.

SangerandEBIoffertrainingattheircampusatHinxtontoover1500earlycareerresearcherseachyear.

Thepotentialandpracticalityofanonlinetrainingmarketplaceandtrainingresourcessharinginfrastructurehasstilltobeinvestigated.TheBBSRCfundedGOBLET(http://mygoblet.org/)andTeSS(http://elixir-uk.org/training-platform)projectsshouldbeinvestigatedforpotentialtransferabilitytootherdomains.

15.Aprocessisputinplacetomakesuretheon-rampcentresandtheNeIworktogethersothatSMEscanactuallyaccessNeIresources.ThisshouldincludeschemestoinduceSMEstouseNeIinfrastructure.

InnovateUKhavesetupthee-InfrastructureSpecialInterestGrouptoaddressthisissue.MembersofIUKandtheKTNaremembersoftheRCUKNeIGroupandthePDGandmeetregularlywithothermembersofbothGroups.

Seehttps://connect.innovateuk.org/vi/web/high-performance-computing

Page 11: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

E-Infrastructurecapitalinvestmentsandmajorinitiatives 11

E-Infrastructurecapitalinvestmentsandmajorinitiatives

The2014SurveydetailedongoinginvestmentsinBigDataAnalyticsProjectsintheareasofHealthInformatics,MedicalInformatics,LifeScienceBioinformatics,AdministrativeDataResearch,AnalysisofPopulationandBusinessData,EnergyEfficientComputing,theSquareKilometreArrayProject,EarthObservation,andDigitalArtsandHumanities.Thisrepresentedinvestmentinexcessof£300mine-Infrastructure.

Manyoftheseprojectshave(oraboutto)comeonlinein2015andcontributetotheUK’sresearchoutputs.

Severalnewinitiativesindatascienceandsimulationhavebeenannounced.ThesestrategicinvestmentsenhancetheUK’sabilityanalyseandmodeldatainkeyeconomicareassuchasHealthandSocietyandWeatherandClimateChange.

Project ResearchOrganisation Amount/£M

CognitiveComputingandDataScience

TheHartreeCentreSTFC(withIBMWatson)

115(BIS)

200(IBM)

AlanTuringInstitute EPSRC,UCL,Edinburgh,Warwick,Oxford,Cambridge

42(BIS)

25(HEIs)

GenomicsEngland NHSandMRC 24

TheMetOffice BIS 97

BigDataandtheInformationEconomy

ESRC 75

TOTAL £378m

£200m(IBM)

Page 12: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

TheStatusoftheRegionalandNationalHPCProjects 12

TherehavealsobeenseveralinvestmentsinlocalHEIprovision,notablyatUCLwith£2mbeinganinvestedina185Gflop/ssystem.Southampton,ImperialandOxfordarealsoconsideringsimilarinvestments.

TheStatusoftheRegionalandNationalHPCProjects

RegionalconsortiaAtalevelabovethelocalinstituteresourcesthereexistsalevelofregionalHPCresource.FiveoftheseregionalHPCcentreswerecreatedwithseedfundingbyEPSRCin2011/2012.Thecurrentstatusoftheseregionalcentresvariesandissummarisedbelow.

Regional ConsortiaHEIs Status

SES5 Oxford,Cambridge,Imperial,Southampton,UCL

AllservicesendJuly2015.

LossofmainUKGPUservice.

Imperial,UCL,OxfordandSouthamptonallhaveplanstoreplace/havereplacedtheseservices.

Lossofcollaborationaroundindustrialengagement

N8 Leeds,Manchester,York,Sheffield,Liverpool,Durham,Newcastle,Lancaster

MainN8Servicehardwaresupportendsinearly2016.

HPCMidlands Loughborough,Leicester SupportuntilJune2016foritsHeraHPCserviceandtheyexpecttocontinuetobeoperationaluntilthen.

ARCHIE-WeSt Strathclyde,Glasgow,GlasgowCaledonian,WestofScotlandandStirling

Hardwaresupportuntil31/3/17.

Hardwarealready3.5yearsold.

MidPlus Warwick,Birmingham,Nottingham,QueenMaryUniversityofLondon

MidPlushassupportagreeduntiltheendofMarch2016

Page 13: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

TheNationalHPCProjects 13

TheNationalHPCProjects

Threeprojectsreceivedsignificantfundingin2011-2012tosetupbaselineHPCresourcesintheUKtosupportsimulationservicesforacademiaandindustry.

• TheDiRACHPCFacility.ThisFacilitybecameoperationalinAutumn2012andisnowthreeyearsold.Itsupportsresearchintheoreticalparticlephysics,solarandplanetaryscience,astrophysicsandnuclearphysics.ItcontainsoneoftheUK’spetaflopsystemsandclustersdesignedtocarryoutdataintensivemodellingandsimulationtasks.ThisFacilitywillbeuncompetitivefromtheAutumnof2015.

• TheHartreeCentre.ThisFacilitybecameoperationalinAutumn2012andisnowthreeyearsold.ItsupportsinnovativesoftwaredevelopmentforIndustryandprovidessimulationservicesforIndustry.ItcontainsoneoftheUK’spetaflopsystemsandhassystemsdesignedtocarryoutdataintensivecomputing.ThisPetaflopFacilitywillbeuncompetitivefromtheAutumnof2015.

• TheNationalServiceArcher.ThisFacilitybecameoperationalinAutumn2013andisnowtwoyearsold.Itsupportssimulationfortheengineering,physicalsciencesandnaturalenvironmentresearchcommunities.ItcontainsthelargesttheUK’spetaflopsystemsandistocarryoutcomputeintensiveresearch.Thisfacilityhasaplannedfiveyearlifespan,terminatingin2018.

• ThethreeFacilitiescontaindifferenthardwareconfigurationsthataretunedtosolveaparticularclassofproblems.WorkloadsfromeachFacilitywouldnotrunoptimallyonanotherFacilityandwouldinterferewiththecoremissionofeachFacility.

BootstrappingtheAlanTuringInstituteWhilsttheAlanTuringInstitutewasstillintheprocessofbeingsetupastheNeIsurveywasbeingundertaken,werecognisethatwithnocapitalallocationforitsownfacilitiestheATIwillbewhollydependentontheNeIforitsdatascienceandcomputerequirements.WithoutasustainablefundingmodelfortheNeI,projectsliketheTuringInstitutewillstruggletogetofftheground,andmaystruggletoachievetheirobjectives–chiefamongstwhichistoimprovetheproductivityoftheUKeconomy.

RCUKshouldasamatterofurgencyprepareabusinessplanforcapitalandoperationalresourcestosupportHPCprovisionintheNeI,indicatingtowhatextentthereductionininternationalcompetitivenessandlossofproductivityduringtheperiod2016-18canbemitigated.ThiswillbenecessaryinordertorealisetheambitionsofinitiativessuchastheTuringInstitute.

Page 14: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

WhoaretheUK’snationale-Infrastructureproviders? 14

WhoaretheUK’snationale-Infrastructureproviders?

TheHEIsremainthemainproviderofdataandcomputeservicestotheNationalE-Infrastructure.37HEIsareprovidingservicesintotheNeI.TheserangefromtheEdinburghParallelComputingService,whichprovidesPetaflopservicestotheNationalHPCserviceandDIRACrespectivelyandpetascaleresearchdataservicestotheUKresearchcommunity,totheUniversityofCranfieldwhichprovidesservicestoengineeringsimulationprojects.

TheseservicessupportbytheHEIsaretheNationalServiceArcher,theResearchDataFacility,DiRAC,GridPPTier2sites,theESRCAdministrativeDataCentreandAdministrativeDataCentres,MRCandCharityfundedMedicalInformatics,theFARRInstitutesHealthInformatics,LifeScienceBioinformatics,theSKA,theRegionalHPCCentres,aswellaslocalprovisionatHEIs.ThesecoverthewholerangeofresearchfundedbytheResearchCouncilsandHEFCE.

JiscprovidesveryhighqualitynetworkservicesbetweenresearchorganisationsviatheJanetnetwork.

TheInternational,CharityandResearchCouncilCentralLaboratoriesprovidespecificlargescaleservicesintheareasofGenomics(SangerInstitute)andBioinformatics(EMBL-EBI),supportforcentralexperimentalfacilities(STFCSCDprovidesservicesforDiamond,GridPPTier1,JASMIN2,ISIS,UKObservatories),SoftwareDataandInnovationservicesforIndustry(theSTFCHartreeCentre).

TheMetOfficeandtheEuropeanCentreforMedium-RangeWeatherForecasts(ECMWF)arethemainproviderofservicestotheweatherforecastingandclimatemodellingcommunities.BoththeseFacilitiesprovidepetascaleservicestotheirusercommunities.

AnewdevelopmenthasbeentheuseofPrivateCloudandPublicCloudservicestoprovideresourcestoNeIusers.Thisiswhollyintheareaofdatafordiscoveryarea.

• GenomicsEnglandhaverentedresourcesfromaUKbasedITprovider• SeveralHEIsandtheMedicalInformaticsprojecteMedLabhaveco-locatedintheInfinityDatacentreat

Slough.Jiscassistedwiththisco-locationproject.AnOpenStackprivatecloudisbeingputinplacetomanagetheseco-locatedservicesinanefficaciousmannertoincreaseresearchoutcomes.

• AnincreasingnumberofHEIbasedprojectsaremakinguseofPublicCloudprovisionandaremakinguseofAmazonWebServices,MicrosoftAzureandGoogleresources.Thescaleofthisusageisveryhardtodetermine;themaindriverisprobablymainlytheuseofvirtualisationtoallowworkflowsthatwerebuiltonlocalworkstationstoruneasilyonlargevirtualclusterswithouttheneedfortimeconsumingportingofworkflowstoclustersintheNeI.

Page 15: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

WhoaretheUK’snationale-Infrastructureproviders? 15

• HEIsaremakingincreaseuseofCloudresourcesforbackofficeandadministrativeservicesusedbytheirorganisations.

Recommendations:

ThenewlyestablishedRCUKCloudWorkingGroupshoulddevelopastrategytoestablishthescopeandpotentialofCloudservicesforresearchandinnovation.ThisshouldidentifywhichpartsoftheNeIcouldusefullyadopthybridCloudtechnologies,andhowCloudtechnologiescouldassisttheNeIinsharingresourcesbetweenitsdifferentprojects.

ToleverageUKinvestmentsinEUandinternationale-InfrastructuressuchastheEuropeanScienceCloud,EGIandEUDAT,RCUKshouldreportonthecurrentlevelofUKparticipationintheseprojectsanddeviseastrategytoensuretheUKbothmaximisesuse,andinfluencesthedesignandmanagement,oftheseinfrastructures.

Page 16: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

SustainablefundingfortheNeIEcosystem

TheHEIProblem

TheadvantageofplacingservicesinHEIsisthatitplacesservicesinthesamelocationastheresearchcommunitiesthatusethem,soallowinggreaterinteractionbetweenserviceproviderandresearchers.

ItalsoprovidesanimportantlocalresourceforresearchersatHEIsandencouragestheentryofnewresearchersandresearchareasintousinglargescaledatafordiscoveryandsimulationservices,byprovidingbothlocalhelpandtraining.

HEIsarealsoallengagedinworkwithpublicsector,businessesandcommerce.TheyareanobviousinterfacewithSMEs.

TheHEIsaretheenginethatwilldrivetakeupofthesenewservicesandthereforeneedtobesuitablyresourced.Howeverthisisdifficulttoachieveinaco-ordinatedfashion.HEIseachhaveaparticularresearchmissionthattheywishtofulfilandtheir5yearplanswillbemarkedlydifferentfromeachother.

Nowthatthe2016-2021sciencecapitalfundingframeworkisinplaceandanE-InfrastructureRoadmapexistsanopportunitypresentsitselftodeviseasustainablefundingframeworkforHEIs.Thisshouldrecognisethat:

1. fECincomeintoHEIsshouldallowHEIstodelivercoresservicestotheirlocalresearchers.2. CapitalmoniesfromRCUKandothersourcescanbeusedtohelptoHEIsprovideservicesforboth

nationalprojectsandspecialistprojects.3. TheexistenceoftheNeISurvey,afundingframeworkandtheroadmapshouldallowNeIprovidersto

self-organiseand• createpartnerships/consortiatoprovideservices• poolandaggregateresourcesviaco-locationormakeuseofcloudservices• createservicesandtheneededcriticalmassofexpertisearoundparticularservicerequirements

orresearchareas• deliveragreaterrangeofservices,particularlyaroundindustrialandpublicsectorengagement,

researchsoftwareengineeringandtrainingservices.4. OperationalcostsremainanissueinthecurrentfundingclimateandHEIscanassistwithmeetingthis

issueifcapitalcanbeusedtofundtheirinfrastructure.

Page 17: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

NonHEIProviders 17

Recommendation:

HPC-SIGhasakeyroletoplayinfacilitatingNeIproviders’self-organisationintopartnerships/consortiatobuildcriticalmassofknowledge,achieveoperatingefficienciesandmaximiseproductivity,creatingeconomiesofscalewithoutcompromisingdiversity.

NonHEIProviders

TheResearchCouncilsandotherresearchorganisationshavemade/willbemakingtheircapitalplansandplanningfortheassociatedoperationalcosts.

Thereismeritinhavingcloserco-operationandknowledgesharingbetweenthoseengagedinprovidingservicesforourlargestprojects.Thiscanleadtoreductionsinprojectoperationalcostsandmakeprojectsmoreproductive.

Recommendations:

AneconomicsustainabilitymodelofthecurrentNeIshouldbeconstructedasamatterofurgencyandasustainabilitystrategydevelopedbythemainparticipantsoftheNeI.Workshopswillbeheldtodevelopthismodelandtodeveloppartnershipsinserviceprovision.ThisisnecessarytoassistwithplanningandtoallowcomparisonofservicecostswithPublicCloudproviders.E-Infrastructureprovidersneedtobegivenconfidencethatthereisalongtermplaniftheyaretoreleasefundsthatwillunderwritestaffrolesbeyondthelifetimeofcurrentprojects,andtodevelopacareerstructureforstaffworkingin“ResearchOperations”(ResOps).

RCUKshouldproposecommonresearchandinnovationrelatedmetricsforallNeIproviderstoensureastandardofreportingtoallowtheeffectiveness,efficiencyanddiversityofNeIserviceprovisiontobeassessed.

Page 18: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

E-Infrastructureasasharedservice

ThepublicationofSirIanDiamond’sreport“Efficiency,effectivenessandvalueformoney”hasservedtoemphasisetheimpressivegainsthattheuniversitysectorhasmadeinimprovingequipmentsharingandengagementwithindustry.TheDiamondreportincludedexamplesofe-Infrastructureequipmentsharing(SES,UKDataService)andhighlightedthevalueofacross-researchcouncilinterestgrouptopromoteassetsharing.

FurtherbigdatainvestmentsbytheResearchCouncilshavecentredonsharedresourcesinregionalfocisuchasN8(LeedsAdvancedResearchComputing)andWales(Farr/ADRC-W).MRCinvestmentsinmedicalbioinformaticshaveresultedindevelopmentofthesharedeMedLabcluster(UCL,QMUL,LSHTM,Crick,Sanger,andEBI).ThisequipmentislocatedintheJiscSharedDatacentretogetherwithMRC-andNIHR-fundedequipmentbelongingtoImperialCollegeandKCL.RelocationofcomputeandstoragefrominadequatepremisesinLondontoamodernenergy-efficientdatacentreallowsHEIstofreeupspaceincentralcampusesandsavemoneyinpowercosts.

Thisconcentrationofco-locatedresourcesislargelyaresultbothoflargecapitalinvestmentswithlittlematchingoperationalcosts,butalsobecausethereisasmallpoolofpeoplewiththetechnicalskillsandknowledgerequiredtooperatetheselargecomputeanddatastoragesystems.Efficiencythroughco-locationisnotjustaboutsavingmoneybutalsoaconsequenceofthelackofsuitableskilledstafftobuildandsupportsystems.Co-locationandpartnershipisneededtoprovideeffectiveserviceprovisiontoacademicandindustrialusers.

ThissurveyhashighlightedthelargeuseofservicesbytheNHSandNHSrelatedprojects.ThisischieflyaconsequenceofnewinvestmentbytheMRCandDoHleadingtothesenewcommunitiesinteractingwiththeNeI.Thesewouldnothaverespondedtothe2014survey.

TheNeIisalreadyworkingtoprovidesecureaccessandsecuredataservicese.g.SafeShareforthesenewcommunities.

JischasestablishedwhatisbelievedtobethelargestVATcostsharinggroupintheUK,withover250institutionsparticipating:https://www.jisc.ac.uk/about/vat-cost-sharing-group.

Recommendations:

ServiceProvidersshouldbeencouragedtoformpartnershipsandpoolresourcestodelivereconomiesofscaleinprovidingafullrangeofservicesacrossacademic,industryandthirdsectororganisations.

Increasetrainingandupdateservicemanagementprotocolsappropriately,sothatthoseusingandmanagingNeIservicesareawareofthesecurityrequirementsoftheirprojectsandtheirsystemsrespectively.Thisshouldbedoneinaproportionatewaythatreflectsdatabeingprocessedanduses/usersofthesystems.

ExplorethepotentialofusingJisc’sVATcostsharinggrouptoreducethefrictionofsharingfacilitiessuchase-Infrastructure,andresearchequipmentmoregenerally.

Page 19: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Industry 19

Industry

Engagementwithindustryisstilllargelyfocusedmostlyinafewsectors-advancedmaterialsandmanufacturing,energyandenvironment,andlifesciences.EngagementisdrivenbythetypeofindustrylocatedclosetotheNeIfacility.LocalEnterprisePartnerships,withtheirspecificexpertiseincertainindustrysectors,shouldprovideanidealroutetobuildrelationshipswithlocalbusinessesspecificallySMEs.

InparticularandelementofNeIinfrastructureshouldbepartoftheinfrastructureplanningbylargescaleregionalinitiativessuchastheNorthernPowerhouse,MidlandsEngine,WalesandScotland.Examplesofthiswouldbe:

• AnorthernJiscSharedDataCentre• ImprovedconnectivityforbusinessandacademiaoutsideoftheSE.

Recommendations

TheELCshouldorganisemeetingsbetweenregionalconsortia/partnershipsandNationalProjectswithLEPsandinitiativessuchastheNorthernPowerhouseandMidlandsEnginetoensurethatlocale-Infrastructureneedsareassessed.

IUKshouldworkwithregionalconsortia/partnershipsandNationalProjectstodeveloptheCatapultsandworkeffectivelywiththeresearchcommunityandgovernment.ExamplesincludethePrecisionMedicineCatapultwhichwillhaveitsHQinCambridgeandtheMedicalTechnologyCatapultwhichbelocatedclosetotheHartreeCentre.

KeyfindingsfromtheDowlingreport(BIS2015)aremirroredintheresponsestothesurvey.Effectivebrokeragebetweenindustry,particularlySMEs,andserviceprovidersiscritical.HEIsinparticularlackresourcetoidentifycommercialopportunitiesincontrasttothelarge/specialistandregionalfacilities(theHartreeCentrehas5dedicatedstaffforbusinessdevelopment).Wherethishasworkedeffectively,ithasbeenduetopartnershipwithinstitutionalbusinessdevelopmentteams.OneexampleisananalysisoftheopportunitiesandbarrierstoSMEuseofregionalcomputeresources,publishedbytheEPSRC-fundedCentreforInnovationinpartnershipwithUCLAdvances.AfurtherchallengeisthelackofawarenessonthepartofindustryoftheavailabilityandtypeofresourcesaspartoftheNeI.

TheDowlingreportalsonotedthatVATwasaparticularobstacletocollaborationandinnovationbetweenacademiaandindustryandrecommendedthat“thegovernmentneedstoaddresstheissueofVATonsharedfacilitiesasamatterofurgency”.

Page 20: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Examplesofindustrycollaborations 20

Recommendations:

IndustryshouldhaveimprovedvisibilityofNeIresources,e.g.viaaccesstodashboardsandportalsdevelopedbytheNeIforitsownuse.

Regionalandnationalfacilitiesshouldhaveanindustrialengagementstrategyincludingmetricsthatpermittheeffectivenessofthestrategytobequantified.

LeverageexistingandnewRCUK-fundedbusinessdevelopmentstafftosupporte-Infrastructurerelatedelementsofindustryengagement,workingwithregionalcentresandlocalHEIs.EncourageuseofstrategicfundingsuchasImpactAccelerationAccountstopump-primebusinessdevelopmentactivitieswithinHEIsspecificallyforcomputeanddataresources.

Examplesofindustrycollaborations

TheEPSRCHPCMidlandssupercomputingcentreofexcellenceisprovidingaccessto£60MofsupercomputingequipmenttoRollsRoyce,facilitatedbyabrokerageschemedevelopedwithJisc.

TheSangerInstitutealsohascollaborations,andprovidesscientificITfor,on-sitespin-outcompaniessuchas14mGenomicsandCongenica.TheGenomeCampus(Hinxton)isbuildinga"BiodataInnovationCentre"nextyear,massivelyexpanding,withspacefor250peopleworkingonindustrial/commercialisationprojects.TheSangerInstitutewillbeprovidingtheITinfrastructurefortheCentre.

TheSquareKilometreArrayScienceDataProcessorprojecthousestheSKAOpenArchitectureLaboratoryinwhichIThardwareandsoftwarecompaniescanworkdirectlywiththeSDPteamtodeveloptechnologiesfortheSKAScienceDataProcessor.Highlightsincludelowenergycomputeclustersfordataprocessing(ARM),veryfastandcheapdiskarraysfordataintensivecomputing(DELL)andusingtheCloudoperatingsystemOpenstacktomanagetheSDPworkflows(Canonical).

OCF,aUKSME,hassupportedanOpenStackusergrouptoaccelerateuptakeofcloudtechnologiesbytheNeIcommunity,includingeMedLab,MRCCLIMBandtheCrickInstitute.

TheUKgovernmenthascommitted£113MtothefutureoftheHartreeCentre.Thisinturnhasleveragedupto£200MoftechnologyandonsiteexpertisefromIBM.FurthercasestudiesareavailablefromtheInnovateUKe-InfrastructureSpecialInterestgroup.

TheUKnowhosts7IntelParallelComputingCentres,whicharedesignedtodrivethedevelopmentofapplicationsandlibrariesonmanycoresystems.ThisisthelargestnumberofCentresoutsideoftheUSAand

Page 21: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

Examplesofindustrycollaborations 21

DiRAChasthelargestnumberofCentresofanyprojectintheworld.Activitiesvaryfromtheportingofapplications,buildingnewapplications,thedevelopmentoftrainingresources,dataimaginganddataanalysis,heterogeneousarchitectures,newmathslibrariesformanycoreprocessorsandthefinegainedmanagementofparalleljobs.TheseactivitieswillallowtheUKtobeamongthefirsttotakeadvantageofmanycoretechnologiesasitmatures.

Seehttp://www.intel.co.uk/content/www/uk/en/processors/xeon-phi/intel-parallel-computing-centers-overview-video.html.

DiRAChoststhreeoftheseCentresattheUniversitiesofEdinburgh,DurhamandCambridge(DAMTP).CentresarealsolocatedattheHartreeCentre,theUniversityofBristol,theEdinburghParallelComputingCentreandmostrecentlyImperialCollege.

TogiveaflavouroftheworkdoneasummaryoftheprojectscarriedoutbyDiRACisgivenbelow.TheseshowthatbyworkingonbeyondtheedgetechnologieswithfirmslikeIntelitispossibletoproducethesekindsofinnovativelowleveltasks/libraries.TheseprojectsalsosupportamixoftraditionalHPCandnewdatascienceapplications.

1.1

1. DevelopedtheGRIDdataparallellibrary,andcodestencils,toalloweasieruseofmatrixmethodsinparallelcodeonmanycoresystems(PeterBoyle,Edinburgh)

2. TestedanddevelopedmanycorevisualisationsoftwareusingXeonPhimanycoresystems-OSPRay(PaulShellard,Cambridge))

3. Developeddataanalyticsanddatamodellingworkflowsusingaheterogeneousprocessor(CPU)andmanycoreXeonPhisystemwhichwasbuiltintheUK(PaulShellard,Cambridge))

4. DevelopedthefinegrainedparalleltaskmanagementQUICKSCHEDlibrary,whichmakespoorlyscalingparallelcodescalemuchbetter.Itmanagestheso-calledloadbalancingproblemandisreallyneededifwewantcodestofunctionatpetascale(RichardBower,Durham).

5. ApplyingnovelstatisticalaccelerationtechniquestoBayesianMethodsandMarkovChainMonteCarlo–thusgreatlyreducingthetimesearchingparameterspace(MarkWilkinson,Leicester)andimprovingquantifiableuncertaintiesinimportantclassesofsimulations.

Page 22: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixA-Listofsurveyorsandacknowledgements 22

AppendixA-Listofsurveyorsandacknowledgements

SusanMorrell(EPSRC)MartinHamilton(Jisc/NeIPDG)ClareJenner(UCL/DiRAC/NeIPDG)JackyPallas(UCL/eMedLab/Farr/NeIPDG)AlanReal(Leeds/N8/HPC-SIG)AndrewRichards(Oxford)JeremyYates(UCL/DiRAC/SKA/NeIPDG)

AppendixB-Whowillreceivethissurvey

TheNationale-InfrastructureProjectDirectorsGroupRCUKNationalE-InfrastructureGroupBISE-InfrastructureLeadershipCouncil.

AppendixC-Listofrespondents

ARCHIE-WeStCardiffUniversityCloudInfrastructureforMicrobialBioinformatics–MRCCLIMB-Birmingham,Cardiff,Swansea,CardiffCranfieldUniversityDiRAC@DurhamUniversityDiRAC@EPCCDiRAC@UniversityofCambridgeDiRAC@UniversityofCambridge(DAMTP)DiRAC@UniversityofCambridge(HPCS)DiRAC@UniversityofLeicesterDurhamUniversity

Page 23: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixC-Listofrespondents 23

EMBL-EBIeMedLab(MRC)-UCL,QMUL,LSHTM,Crick,Sanger,EBIEPCCFarrNorth,HealtheResearchCentreHPCMidlandsHPCWalesImperialCollegeLondonKing'sCollegeLondonLancasterUniversityLoughboroughUniversityN8HPCNBIPartnershipLtdNorwichBioscienceInstitutes(TGAC,JIC,IFR,TSL)QueensUniversityofBelfastSES/CFISTFCHartreeCentreSTFCScientificComputingDivisionTheFrancisCrickInstituteTheInstituteofCancerResearchTheUniversityofBirminghamTheUniversityofNottinghamTheUniversityofSheffieldUVRI/MRCMedicalInformaticsCentreUniversityCollegeLondonUniversityofAberdeenUniversityofBathUniversityofBristolUniversityofCambridgeUniversityofEdinburghUniversityofExeterUniversityofGlasgowUniversityofLeedsUniversityofLeicesterUniversityofLiverpoolUniversityofManchesterUniversityofOxfordUniversityofPortsmouthUniversityofSouthamptonUniversityofStAndrewsUniversityofSussexUniversityofWarwickWellcomeTrustSangerInstitute

Page 24: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixD–Servicemanagement:Thesurveyquestions 24

AppendixD–Servicemanagement:Thesurveyquestions

Q1OrganisationnameQ2OrganisationalunitQ3EmailaddressQ4Jobtitle

Budget

Q5IsyourHPCbudgetringfenced,ordoyouhavetomakeafreshcaseforsupporteachtime?Q6IsyourprimarybudgetforHPCCAPEXorOPEX?Q7AreyoudirectlychargedforpowerandcoolingorotherEstatescosts?Q8DoyoucapitaliseEstatescosts,orpaythemfromyourOPEXbudget?Q9Tellusaboutyourdatacentre(s)(selectallthatapply)Q10DatacentrePUE(ifknown)Q11Datacentrepowerdraw,ifknown

Projectmanagement

Q12Howistheprojectmanagementneededtoimplementanddelivernewprojectsprovidedatyourinstitution?Q13HowareProjectManagement(NewServiceImplementationandDelivery)resourcescalculated?Q14DoyouworkouttheresourcesneededintermsofFTE?Q15WhichofthefollowingbestapplytoyourHPC/BigData/ResearchComputing/ScientificComputingActivity?Q16DoyouusemetricstoworkouttheProjectManagementresourcesneeded?[itisassumedthesearechargedagainsttheCapitalCostoftheProject]Q17Arethey...[tickboxesforpercentagerangeofcapitalcostduetoProjectManagement]Q18WhichofthestatementsbestdescribehowyouarriveattheProjectManagementresourcesneededtodeliverandimplementtheproject?

Staffing

Q19HowmanyFTEsupportyourHPCactivity?Q20FTEcountif>4Q21HowmanywomenareinvolvedinHPCserviceprovisionatyourorganisation?Q22WhatmodelofHPCprovisionhasyourorganisationadopted?Q23BreakdownofeffortbyFTEQ24WomeninHPCservicemanagement(FTE)Q25Whatisyourapproachtotraining?PleasetickallthatapplyQ26URLforfurtherinformationontrainingifapplicableQ27Howwouldyouprefertopooltrainingacrossthesector?

Page 25: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixD–Servicemanagement:Thesurveyquestions 25

Q28Whatproportionofyourresearchersareinself-supportingresearchgroups?Q29Whatarethemajortrainingchallengesandwhichdoyouaddresslocally?Q30OthertrainingchallengesQ31Otherformsofsupport.Doesyourorganisationdoanyofthefollowing?Q32AreyouabletoprovideHPCcasestudiesfor(e.g.)HPC-SIGwebsite,RCUKandInnovateUKe-InfrastructureSIG?

CloudandSharedServices

Q33Whichstatementsbestcharacteriseyourorganisation'suseof"cloud"services?Q34Useofcloudservices-furtherinformationQ35Whichstatementbestcharacterisesyourorganisation'sapproachtoHPCasasharedservice?Q36WhichareasofHPCprovisionmightyoushare?

Researchdatamanagement

Q37DoyouhaveaResearchDataManagementpolicy?Q38DoesyourorganisationhaveaResearchDataManagementservice?Q39URLforfurtherinformationaboutResearchDataManagementatyourorganisationifapplicableQ40WhattechnologieshaveyoudeployedthatcanberegardedasbeingforDataExploration(“BigData”)activities?

Academicimpact

Q41WhatareyourKeyPerformanceIndicatorstomeasureacademicimpact?Q42RefereedandconferencepaperspublishedQ43URLlistingthesepublicationsifapplicableQ44DoesyourorganisationproduceanAnnualReport?Q45DoesyourorganisationpublishResearchHighlights?

Whousesyourservice?

Q46DoyouprovideHPCservicesbeyondyourimmediateorganisation?Q47Whatservicesdoyouprovidetothirdparties?Q48HowmanyHEIsresearchgroupsuseyourservices?Q49PublicandthirdsectororganizationsQ50Whatpercentageofyourusersarefromindustry?Q51Ofindustryusers,whatproportionareSMEs?Q52WhatpercentageofsystemtimeisbeingusedbySMEs?Q53AreyoutakinganyspecificstepstoincreaseSMEuptake?Q54Whatsectorsarebeingservedbyyourindustrialusers?Q55Pleaseusethisspaceforanyfurtherinformationyouwouldliketoprovide

Page 26: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

AppendixE–Servicemanagement:Summaryofthesurveydata

Wherefeasiblewehavetriedtobreakdownsurveyresponsesbylargeandspecialistfacilities,regionalcentres,andhighereducationinstitutions.Eventhen,manyHEIsclearlyoperateonashoestringe.g.withminimalstaffing,whereasothershavelargeteamsworkingonscientificcomputing-soitisnottrivialtodrawcomparisonsbetweenthethreegroups.WehaveincludedHPCWalesasa"regional"facilitybecauseofitsprimaryfocusonWelshresearchersandindustry.ItshouldbenotedthatHPCWaleswasnotpartoftheEPSRCregionalsupercomputercentreinitiative.

Budgetingmodel-Thevastmajorityofrespondents(73%ofthosewhoansweredthisquestion)indicatedthattheirprimarybudgetfore-Infrastructurewascapital(CAPEX).However,whilst63%statedthattheywerenotchargedforpowerandcoolingorotherEstatescosts,28%ofrespondentsindicatedthattheypaidEstatescostsfromOPEX-implyingthatsomeelementsoftheserviceprovisionbeyondsalarieswerebeingcoveredviaOPEX.

Thirdpartydatacentres-TheonlythirdpartydatacentrethatrespondentsreportedusingwastheJiscSharedDataCentre,inuseby8%ofinstitutionsresponding.Ofthoserunningin-housedatacentres,moststatedaPUEofbelow1.5.Severalrespondentsindicatedthattheywereonlyabletoreportdatacentrepowerdrawforabuildingoranentiredatacentrewhichtheirfacilitywaspartof.

Projectmanagementapproach-Itwasrareforrespondentstocallonthirdpartyprojectmanagementservicesfornewserviceimplementationanddelivery,withthisalmostalwaysbeingdonein-house,sometimes(13%ofcases)viadedicatedprojectmanagersbasedinITServicesdepartments.Respondentshadarangeofapproachestocalculatingtheirresourcerequirementsforamajore-Infrastructureproject,rangingfromhighlyformalisedto"backoftheenvelope".

68%ofrespondentsstatedthattheycalculatedstaffingcostsfortheirimplementationprojects.Itwasnotunusual(36%ofrespondents)forinstitutionstobidforadditionalstaffingaspartofane-Infrastructureprojectproposal.Onlyoneinstitutionusedmetricstoworkoutprojectmanagementresourcesneeded,and46%ofrespondentsagreedwiththestatementthat"wedon'tcalculatetheseresources,itjusthappens".

Staffing-24%ofrespondents(includingHEIsandlargeandspecialistfacilities)statedthattheyhadtwoorlessfulltimeequivalentemployeessupportingtheire-Infrastructureactivities.Conversely,9out25HEIsresponding(36%)hadteamsoffourormoreemployeesworkinginscientificcomputing.OneHEIreported23FTEsworkingonitsservices,buttheselargerstaffingcomplementsweretypicallyassociatedwithregionalandlarge/specialistfacilities.

WomeninHPC-thesurveyshowedthattherewere55womeninvolvedinserviceprovisionacrossallofthefacilities,mainlyworkingparttime.ThesituationwasparticularlydisappointinginHEIs,withonly14womeninvolvedinserviceprovisionininstitutions.Manyrespondentsstatedthattheywereunsurehowmanywomenwereinvolvedintheirservices,whichisoddgiventhesmallteamsizesatalargeproportionofHEIs.AseparatesurveyhasbeenconductedofARCHERandDiRACserviceusers,togaugethesituationfromtheuser'sperspective.

Page 27: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixE–Servicemanagement:Summaryofthesurveydata 27

Training-36%ofrespondentsreportedthattheycarriedoutintroductorytrainingcourses.Thiscoversjust40%ofHEIrespondents.Only24%ofHEIsrespondingoperatedanadvancedtrainingcourse,andonly34%ofHEIsactivelypromotedtrainingavailablefromothersourcese.g.ARCHERandapplicationvendors.OnlytwoinstitutionsprovidedtrainingaspartofaDoctoralTrainingProgramme,and3aspartofaCentreforDoctoralTraining.However,therewaswidespreadinterestinsharingtrainingmaterialsfreelyonline(73%ofrespondents)and/orcreatingamarketplacefortrainingmaterialsanddeliveryproviders(34%ofrespondents).

Skills-10outof25HEIsresponding(40%)reportedthatlessthanhalfoftheiruserswereinself-supportingresearchgroups.HEIs,regionalcentresandlarge/specialistprovidersallsawarangeoftrainingchallengesratherthanasinglekeyissue-achangefromthe2014surveywhereLinuxorientationemergedasahugeproblem.WewillfollowthisdevelopmentupwithHPC-SIGmemberstoseewhethertherequirementforLinuxskillshasdiminished,oritisbecomingmorecommonfornewserviceuserstoalreadybefamiliarwithLinux.

Communitybuilding-Institutionsreportedthattheywereaugmentingtrainingwitharangeofadditionalcommunityandexpertisebuildingapproaches,includinginformalnetworkingforserviceusers(68%ofHEIs),wikisoronlineforums(48%ofHEIs),andpublicspeakingatresearcherfocussedevents(40%ofHEIs)ordedicatedscientificcomputingevents(36%ofHEIs).Manyserviceproviders(24%ofHEIsand50%oflargeandspecialistfacilities)werealsoabletoprovidecasestudiesofhowtheirservicehadbeenused.

ResearchDataManagement-76%ofrespondentssaidthattheyhadaResearchDataManagementpolicy,with4HEIsreplyingthattheywereeitherunsureaboutthisordidnothaveapolicyatpresent.51%ofrespondentshadapilotorproductionRDMserviceatthetimeofrespondingtothesurvey,with5unsure.

Academicimpact-commonKeyPerformanceIndicatorswerethenumberofusersoftheservice(63%ofrespondents),thenumberofresearchpaperspublished(61%),thenumberofprojectsrunonsystems(51%)andtheamountofresearchgrantfundingbroughtin(51%).However,onlyahandfulofrespondents(e.g.4HEIs)wereabletolistthenumbersofpublishedpapersthathadusedtheirsystem,andonlyasmallnumberofrespondentswereproducinglistsofpublishedpapers,anannualreportorresearchhighlights.

Crosssectorandindustrialimpact-manyrespondentswereinvolvedininter-institutionalcollaborationssuchastheEPSRCregionalcentres,theFarrInstitute,GridPPandtheMedicalBioinformaticsInitiative.51%ofrespondents,including32%ofHEIs,providedcomputeandstorageservicestothirdparties.27%ofrespondentsworkedwiththeNationalHealthService.Industrialuseofnationale-Infrastructurefacilitieswaslargelyconfinedtoasmallnumberofproviders,withsomeHEIsoutstrippinglargeandspecialistandregionalcentres.Just7providersreportedthattheywereworkingwithSmalltoMediumsizeEnterprises(SMEs).

Page 28: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 28

AppendixF–Servicemanagement:FullbreakdownofsurveydataBudget

Q5IsyourHPCbudgetringfenced,ordoyouhavetomakeafreshcaseforsupporteachtime?

HEILarge/

Specialist Regional GrandTotal

FreshcasetoOrganisation 12

12

Partlyringfenced 8 2 1 11

FreshcasetoExternalFunders

5 1 6

Whollyringfenced 4 2

6

Other 1 3 1 5

GrandTotal 25 12 3 40

Q6IsyourprimarybudgetforHPCCAPEXorOPEX?

HEI Large/Specialist Regional GrandTotal

CAPEX 18 9 2 29

Other 3 2 1 6

OPEX 4 1 5

GrandTotal 25 12 3 40

Page 29: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 29

Q7AreyoudirectlychargedforpowerandcoolingorotherEstatescosts?

HEI Large/Specialist Regional GrandTotal

No 19 4 2 25

Yes 5 5 1 11

Other

3

3

Don'tknow 1

1

GrandTotal 25 12 3 40

Q8DoyoucapitaliseEstatescosts,orpaythemfromyourOPEXbudget?

HEI Large/Specialist Regional GrandTotal

19 7 2 28

OPEX 6 4 1 11

Other

1

1

GrandTotal 25 12 3 40

Q9Tellusaboutyourdatacentre(s)(selectallthatapply)

HEILarge/

Specialist Regional GrandTotal

Weoperatein-housedatacentres

24 12 3 39

WeusetheJiscDataCentre 1 2

3

Weuseathirdpartydatacentre

GrandTotal 25 14 3 42

Page 30: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 30

Q10DatacentrePUE(ifknown) Q11Datacentrepowerdraw,ifknown

Projectmanagement

Q12Howistheprojectmanagementneededtoimplementanddelivernewprojectsprovidedatyourinstitution?

HEI Large/Specialist Regional GrandTotal

Wedosomeaspectsoftheprojectmanagementusingourin-housesystemsteamsandtherestisprovidedbythesuccessfulvendor

12 4 3 19

WedoALLaspectsoftheprojectmanagementusingonlyourin-housesystemsteams

6 7

13

OurcentralITdept.hasprojectmanagementstaffandwecanusethemifweneedto

4 1

5

WepayexternalcontractorstohandleALLaspectsoftheprojectmanagement

1

1

GrandTotal 23 12 3 38

Q13HowareProjectManagement(NewServiceImplementationandDelivery)resourcescalculated?

• SupportedbyanSTFCgrantandarebundledwiththeoverallDiRACproject• ProjectManagementresourcesarecalculatedonaneedsbasis,andcurrentlywerequireoneFTE,whichisfunded

fromtwodifferentsources• Wemakeanestimateofallstaffeffort(FTEs)foreachproject,attheprojectproposalstage.• Weuseour25yearsexperiencewhenwritingproposalsandtenderstoprovideappropriateresourcesforeachproject

01000200030004000500060007000

Q11.Datapowerdraw(kW)

Large/specialist

Regional

HEI

0

0.5

1

1.5

2

2.5

Q10.DatacentrePUE

Large/specialist

Regional

HEI

Page 31: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 31

andservice.• Weareattheinitialstart-upphase,so100%ofthetimehasthusfarbeenonnewserviceimplementationand

delivery.• BytheBusiness&Deliveryteams,baseduponanextensivespreadsheet• Basedonourexperienceandexpertiseandthecustomerrequirements• BasedonstandardFTEcosts• Veryinformally;foreachprocurement,oneoftheHPCarchitectswilltakeonprojectmanagementresponsibilityfor

thenewservice.• EachProjectandassociatedservicesaresubjecttoaninitialreviewbytheOperationalandUserServicesGroup• ProjectmanagementcostsfortheimplementationofanewHPCserviceareincludedinthesystemprocurement• Nodirectchargingsincetheseareinternalresources• Itisbasedonthesizeofthecontractandestimateddeploymenttime• Dayratebasedonactualcostsblendedacrossamixofcorepermanentstaffandcontractors• FormalProjectManagementisonlyrequiredonanadhocbasissotheresourcecalculationsaresimilarlydoneonan

adhocandcasebycase• Aspartofourprojectdeliveryprocesses.• Advisefromin-houseprojectdeliveryteams,typically1PMpermajorproject.• PartoftheSysAdminsjob• Resourcescalculatedinternally

Q14DoyouworkouttheresourcesneededintermsofFTE?[ItisassumedthesearechargedagainsttheCapitalCostoftheProject]

HEI Large/Specialist

Regional GrandTotal

No 6 1

7

Yes 6 8 1 15

GrandTotal 12 9 1 22

Q15WhichofthefollowingbestapplytoyourHPC/BigData/ResearchComputing/ScientificComputingActivity?

HEILarge/

Specialist Regional GrandTotal

TheyarecalculatedintermsofrequiredFTEandsomeofthisistensionedagainstouroverallstaffresourceallocationandtherestisrequestedintheProjectTender.

2 5 1 8

Page 32: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 32

TheyarecalculatedintermsofrequiredFTEandthisistensionedagainstouroverallstaffresourceallocation.

3 3

6

Unsure 2 2

4

TheyarecalculatedintermsofrequiredFTEandsomeofthisistensionedagainstouroverallstaffresourceallocationandsomeisrequestedfromourcentralITdept.

1

1 2

TheyarecalculatedintermsofrequiredFTEandallofthisHAStobeprovidedbyourcentralITdept.

1

1

TheyarecalculatedintermsofrequiredFTEandallofthisisrequestedfromourcentralITdept.

1

1

GrandTotal 10 10 2 22

Q16DoyouusemetricstoworkouttheProjectManagementresourcesneeded?[itisassumedthesearechargedagainsttheCapitalCostoftheProject]

HEI Large/Specialist Regional GrandTotal

No 9 2 1 12

Yes 1 1

GrandTotal 10 2 1 13

Q17Arethey...

Lessthan5%ofthetotalcapitalcostsBetween5and9.99%ofthetotalcapitalcostsBetween10and14.99%ofthetotalcapitalcostsBetween15and19.99%ofthetotalcapitalcostsOver19.99%ofthetotalcapitalcostsUnsure

Oneinstitutionrespondedtosaythattheirprojectmanagementcostswerelessthan5%oftotalcapitalcosts.

Page 33: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 33

Q18WhichofthestatementsbestdescribehowyouarriveattheProjectManagementresourcesneededtodeliverandimplementtheproject?

HEILarge/

Specialist Regional GrandTotal

Wedon'tcalculatetheseresources-itjusthappens

4 1 1 6

Wedon'tcalculatetheseresourcesbecausetheseresourcesarealreadyincludedinouroverallservicedeliveryplans.

4 1

5

Alreadyincluded 1 1

Unsure 1 1

GrandTotal 10 2 1 13

Staffing

Q19HowmanyFTEsupportyourHPCactivity?Pleasechoosethenearestanswerandentertheexactfigurebelowif>4

HEILarge/

Specialist Regional GrandTotal

1 2 1 3

1.5 3 1 4

2 2 2

2.5 5 1 1 7

3 2 3 5

3.5 1 1

4 3 1 4

>4 6 5 11

GrandTotal 24 11 2 37

Page 34: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 34

Q20FTEcountif>4 Q21HowmanywomenareinvolvedinHPCserviceprovisionatyourorganisation?

Q22WhatmodelofHPCprovisionhasyourorganisationadopted?

HEI Large/Specialist

Regional GrandTotal

CentralITdepartment 2 3 5

DevolvedtoFacultiesandSchools

11

Governmentresearchestablishment

2 2

Independentcentre 2 2

Mixtureofdevolvedandcentral

4 4

Other 1 1

Regionalornationalfacility 1 1 2

GrandTotal 7 8 2 17

0

10

20

30

40

50

60

70

80

Q20.FTEcountif>4

Large/specialist

Regional

HEI

0

5

10

15

20

Q21.WomeninHPCprovision

Large/specialist

Regional

HEI

Page 35: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 35

Q23BreakdownofeffortbyFTE

89

46

42

13

1

02468

10

Q23.Effortforsystemsupport

13

41 1

41

0

5

10

15

Upto0.5FTE

0.5-1FTE

1to1.5FTE

3to4FTE

Over5FTE

Unsure

Q23.Effortforsonwareengineering

20

41 2 2 2 1

0510152025

Q23.Effortforusertraining

18

4 31 2 2 1

0

5

10

15

20

Q23.Effortforprojectmanagement

22

4 3 2 1 2 10510152025

Q23.Effortforstrategicengagement

16

7 52 2 1

41

05

101520

Q23.Effortforapplicaoonsupport

Page 36: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 36

Q24WomeninHPCservicemanagement(FTE)

5

32

1

3

0123456

Upto0.5FTE

0.5to1FTE

1to1.5FTE

1.5to2FTE

Unsure

Q24.Womeninsystemsupport

3

2

1 1

3

00.51

1.52

2.53

3.5

Upto0.5FTE

1to1.5FTE

3to4FTEOver5FTE Unsure

Q24.Womeninsonwareengineering

6

21 1

3

0

2

4

6

8

Upto0.5FTE

1to1.5FTE

2to3FTE

Over5FTE

Unsure

Q24.Womeninapplicaoonsupport

6

2 21

2

0

2

4

6

8

Upto0.5FTE

0.5to1FTE

1to1.5FTE

3to4FTE

Unsure

Q24.Womeninusertraining

16

1 1 1 1 3 2

05101520

Q23.Effortforotherusersupport

Page 37: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 37

TrainingandUserSupport

Q25Whatisyourapproachtotraining?Pleasetickallthatapply

HEI Large/Specialist Regional GrandTotal

Werepurposeexistingtrainingmaterialsdevelopedelsewhere

12 5 2 19

Wedevelopourowntraining 9 4 2 15

Werunourownintroductorytrainingcourses

10 3 2 15

Wepromotetrainingavailablethroughe.g.ArcherCSEandapplicationvendors

8 2 1 11

Werunourownadvancedtrainingcourses

6 4 0 10

8

1 12

0

2

4

6

8

10

Upto0.5FTE

1.5to2FTE

4to5FTE Unsure

Q24.Womeninotherusersupport

9

31 1

3

0246810

Upto0.5FTE

0.5to1FTE

1to1.5FTE

3to4FTE

Unsure

Q24.Womeninstrategicengagement

5

21 1

3

0

2

4

6

Upto0.5FTE

0.5to1FTE

1to1.5FTE

4to5FTE

Unsure

Q24.Womeninprojectmanagement

Page 38: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 38

WeprovidetrainingaspartofaCentreforDoctoralTraining 3 1 1 5

WeprovidetrainingaspartofaDoctoralTrainingProgramme 2 1 1 4

Other 0 1 1 2

Wedonotprovidetraining 0 0 0 0

Q26URLforfurtherinformationontrainingifapplicable

• https://virgodb.cosma.dur.ac.uk• http://www.dirac.ac.uk/training.html• https://www.epcc.ed.ac.uk/education-training,http://archer-www.epcc.ed.ac.uk/training/• http://www.hpcwales.co.uk/solutions/skills-and-training• http://www.shef.ac.uk/cics/research/training• http://www.cardiff.ac.uk/arcca/services/events/index.html• http://www.kcl.ac.uk/hpc/services/training.aspx• http://www.ucl.ac.uk/isd/services/research-it/training• https://www.acrc.bris.ac.uk/acrc/training.htm• https://www.wiki.ed.ac.uk/display/ecdfwiki/Courses+and+Events• http://wiki.rac.manchester.ac.uk/community/Courses• http://www.arc.ox.ac.uk/content/training• Werunourownintroductorytrainingcourses• http://www.sussex.ac.uk/its

Q27Howwouldyouprefertopooltrainingacrossthesector?

HEILarge/

Specialist Regional GrandTotal

Sharematerialsonlinefreelyanddeliverlocally

21 8 1 30

Marketplaceoftrainingmaterialsandcoursedeliveryproviders 8 4 2 14

Other... 1 1 1 3

Donotfeelthiswouldbebeneficial

1 2 0 3

Page 39: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 39

Q28Whatproportionofyourresearchersareinself-supportingresearchgroups?

HEI Large/Specialist Regional GrandTotal

Noneornotknown

6 1

7

<=25% 4

4

25%to50% 6 2 2 10

50%to75% 3 1 1 5

>75% 6 7

13

GrandTotal 25 11 3 39

Q29Whatarethemajortrainingchallengesandwhichdoyouaddresslocally?

HEILarge/

Specialist Regional GrandTotal

Generalintroductiontoprogramming

2 4 2 4

Targettedprogrammingadvice,e.g.usingMPI,PE

3 4 2 4

Datascience 4 4 2 4

Other.. 4 3 1 4

Linuxorientation 1 3 1 3

Applicationspecificadvice 3 3 2 3

Q30Othertrainingchallenges

• Codeoptimizationandportingtonewtechnologies.• AswellasaddressinglocaltrainingchallengesweaimtospreadtrainingexpertiseacrosstheUKandbeyond.All

materialsareCClicensed,weengagewithothercentrestohelptraintheirtrainers,wedeveloponlinetraining

Page 40: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 40

materialsthatcanaccessedfromanywhereatanytime.• Trainingthetrainersi.e.enablingscientificcommunitiestosupportthemselves.• Bioinformaticsandgenomicsskillsforscientists.Resourcingpressurehasmadethisdifficulttoprovidelocally.• ContinualchallengeastothebestROI-tensionbetweentargetingthegrowthofnewusercommunities(resource

intensive)andfocusingontheestablishedresearchersandenhancingtheiroutput(publicationsetc.)-thelatterrequiresdomainexpertisebythetrainers.

• WhethertoimplementchargingtensionedagainstuptakebyinitialresearcherinterestHowtobestmonitortheimpactofthetrainingthroughsubsequentfollow-up.Multi-stagequestionnaire/survey-immediatelyposttrainingand3monthsafter.

• TrainingbiologiststouseGalaxy.• IntroductiontotheHPCcluster-addressingscheduler,storageandbestpractices• Helpinguserscopewiththechangesinworkflowinmovingfromdesktopcomputingtoamorebatchoriented;central

HPCsystem,includingtheprogressionfromGUIstoamorecommandlineenvironment.Thiscantakeagooddealofeffortwithsomenewusers.

• DeliveringwhatlookslikebespoketrainingtoCDTsisachallenge.Asiscoordinatingregionalandnationaleffortstoshareresources.Inrealityweprovideabroadrangeoftrainingfrommanydifferentsources.Wealsoembracetrainingdeliveredbyvendorstoourresearchers.

• Informationaboutavailablesoftware(incl.visualisationandanalysis).Examplebatchjobs.Occasionallyface-to-facetraining.

Q31Otherformsofsupport

HEILarge/

Specialist Regional GrandTotal

InformalnetworkingforexistingHPCusers

17 7 3 27

HPCcommunitywiki/forum 12 10 2 24

SeminarseriesfeaturingHPCbasedresearch

9 4 2 15

Speak/exhibitatresearcherfocussedevents

10 3 2 15

EmbedHPCstaffinresearchgroups

7 7 0 14

DirectworkonresearchprojectsbyHPCstaff

8 6 0 14

Other. 2 3 0 5

Page 41: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 41

Q32AreyouabletoprovideHPCcasestudiesfor(e.g.)HPC-SIGwebsite,RCUKandInnovateUKe-InfrastructureSIG?

HEILarge/

Specialist Regional GrandTotal

No 16 5 21

Yes 6 6 3 15

GrandTotal 22 11 3 36

CloudandSharedServices

Q33Whichstatementsbestcharacteriseyourorganisation'suseof"cloud"services?

HEILarge/

Specialist Regional GrandTotal

Someofourresearchersusecloudservicesinanunsupportedway 14 4 1 19

Wedon'tusecloudservicesatall 6 2 2 10

Someofourresearchersusecloudserviceswithcentralsupport

4 3 0 7

Other.... 2 3 1 6

Wehaveapolicyonuseofcloudfacilities

0 1 0 1

Thereareothercloudproviderswewouldliketoseethesectorpartnerwith 1 0 0 1

WeuseJisc'sclouddeals,e.g.Amazonportal

1 0 0 1

Mostofourresearchersusecloudserviceswithcentralsupport

0 0 0 0

Page 42: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 42

Weexclusivelyusecloudbasedcomputefacilities 0 0 0 0

Q34Useofcloudservices-furtherinformation.Pleasedescribethecloudservicesyourorganisationuses,orwishestouse.

• Weusegithubforsoftwareversioncontrolandpromoteits(orequivalentalternativeservice)toourusers.Manyusersusecloudservicesontheirown,too,e.g.dropboxetctosharefilesbutthisistotallyunsupportedandnotrecommended.

• Diracfacilitiesareprovided"asaservice"fromthepointofviewoftheendusers.AlthoughthisisnotususingHPCasaService,itisusprovidingHPCasaService.

• EMBL-EBIEmbassyCloud-VMware&OpenStack• http://www.fortissimo-project.eu/Webservicesforbigdata:EUDATInvestigatinguseofStarClusterforspecificHPC

workloadsinthecloud• OccasionalAmazonS3atsmallscale.VMsforad-hocwebservers(shortlived).HostedVMsfordeliveringoff-site

training.• WeprovideprivatecloudtoouruserbaseusingavarietyoftechnologiessuchasVCloud,OpenNebula• LargescaleuseofOpenStackforresearch.VMWareusedtosupportbusinessactivities.• AWSatthemoment.We'dliketobeusingOpenStackorsimilarourselves,toprovideanAWS-likeexperiencetothe

users.• WehavemadeanumberofeffortstolaunchourownCloudservicestogetherwithFujitsu.Thisrequiresanenhanced

HPCenvironment(security,easeofusewithdomainspecificportals,billingandSLAcontractualexpertise)thatarenottypicallynotrequiredbyanacademic

• WehaveevaluatedAWS(includingextendingourclusterusingBrightclustermanager),butfounditunsuitableformanyHPCapplications.

• Somead-hocusageofAmazoncloudviaComputerScienceresearchergroups.UndertakingacostandbenchmarkingperformanceanalysisincollaborationwithBioScienceinQ3/2015Annualcostperformanceanalysisofcloudvs.in-houseprovisionpresentedtoGovernanceGroup

• Wewouldbeinterestedinsomeformofcloud-burstingforcompute-orpossiblySaaSforsomeproductswhichwouldrelievethemainHPCsystemofthesesmallerusers.

• WearealsoaspecialistHPCcloudserviceinourownright.• Someresearches(e.g.computerscientists)useIaaScloudservicesinanunsupportedway.Wehavealsointerfaced

ourHPCtoAmazonAWSfor"cloudbursting"experimentallybuthavenotfoundtheperformance/priceratiotobebeneficialatthistimeforrawHPC.

• Office365,Azure,Simplicity• AmazonGlacier.Arkivum.• Weusecloudforthingslikeemail&someworkingdatastore.We'reinconversationwithMicrosoftabout

opportunitiesinscalar/throughputprocessing.• Weareinterestedinexploringwhatadditionalfeaturesofourexistingservices,oradditionalservices,cloudresources

wouldenable.• WeareinvestigatingtheeasewithwhichuserscanpotentiallyexploittheAmazonportalviatheJiscdeal.• Amazon,MSAzureetc.Theuseoftheseservicesismostlyunsupportedbutwehaveanapproachthatgivessupport

toresearchnomatterwhatplatformtheyuse.Cloudispartofthat.

Page 43: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 43

• UseofAmazon,MSAzure-especiallywherefreeaccesshasbeengrantedtosupportprojects.In-house'cloud'typeservicesusedfordeliveryofsomeHPCrelatedcomputations-mainlyfromVMWarebasedsystems

• GoogleDrive• Developingprivatecloudandcloudbursting.• Local/privatecloud;wedonotusepublicclouds(atleastnotinacentrally-supportedway)

Q35Whichstatementbestcharacterisesyourorganisation'sapproachtoHPCasasharedservice?

HEI Large/Specialist Regional GrandTotal

WeprovideaccesstoHPCasasharedservice,e.g.EPSRCregionalHPCcentres

7 9 3 19

Wemaybeinterestedinexploringsharedservicesinthefuture 6 1

7

Notapplicable 3 1 4

Weusesharedservicesprovidedforusbyothers 4

4

Other 3 3

Q36WhichareasofHPCprovisionmightyoushare?

HEILarge/

Specialist Regional GrandTotal

Training 12 4 3 19

Applicationsupport 10 5 3 18

Systemmanagement 10 6 0 16

End-to-endserviceprovision 7 5 2 14

Datacentre 5 7 1 13

Softwaredevelopmentassistance

8 4 1 13

Page 44: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 44

Otherusersupport 7 4 1 12

Other 1 1 0 2

Researchdatamanagement

Q37DoyouhaveaResearchDataManagementpolicy?

HEI Large/Specialist Regional GrandTotal

No 2 1 3

Unsure 2 3 5

Yes 21 7 3 31

GrandTotal 25 11 3 39

Q38DoesyourorganisationhaveaResearchDataManagementservice?

HEI Large/Specialist

Regional GrandTotal

Notyet 6 6 12

Pilotservice 9 1 10

Productionservice 6 3 2 11

Unsure 3 2 5

GrandTotal 24 11 3 38

Q39URLforfurtherinformationaboutResearchDataManagementatyourorganisationifapplicable

• http://www.lib.cam.ac.uk/dataman/• https://www.epcc.ed.ac.uk/facilities/uk-research-data-facility• https://www.stfc.ac.uk/1930.aspx• AtthemomentweadoptRDMpracticesfromourstakeholderinstitutions.• http://www.strath.ac.uk/researchdataproject/• http://www.sheffield.ac.uk/library/rdm

Page 45: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 45

• http://www.nottingham.ac.uk/research/research-data-management/index.aspx• http://www.cardiff.ac.uk/insrv/researchdata/managingdata/index.html(Converis-basedsolution)• http://www.kcl.ac.uk/library/researchsupport/research-data-management/index.aspx• http://www.lancaster.ac.uk/library/rdm/• http://www.lboro.ac.uk/service/research/offcampus/rdm.htm• https://www.qub.ac.uk/directorates/ResearchEnterprise/ResearchPolicy/ResearchDataManagementPolicy/• http://www.bath.ac.uk/research/data/• http://data.bris.ac.uk/• http://www.ed.ac.uk/schools-departments/information-services/research-support/data-management• http://www.liv.ac.uk/csd/research-data-management/• http://tinyurl.com/owpwj2o• http://researchdata.ox.ac.uk/• http://www.st-andrews.ac.uk/itsupport/academic/research/about/strategy/• http://www.sussex.ac.uk/library/research/researchdatamanagement/

Q40WhattechnologieshaveyoudeployedthatcanberegardedasbeingforDataExploration(“BigData”)activities?

• Largesharedmemorycomputers;100-terabyte-scalefile-systems• HTE,Spark/Hadoop• DIRMachine(DataIntensiveVMCluster)RDFCluster(LinuxContainersbasedclusterforDataAnalytics)SPRINT(MPI

ParallelR,http://www.r-sprint.org/)SGIUV20• 00(DigitalHealthInstitute,forcomplexdataanalytics• IBMproducts:BigInsights,Streams,InfosphereDataExplorer,SPSS,InfosphereContentAnalytics,Cognos,Watson• SCDmanageslargescalescientificdatainorderof50PBwhichisaccessibleathighdataratesusingarangeof

technologiesappropriatetothecustomer.e.g.for• JASMINusingPanasaswithamultiterabitIOcapacity.GridPPCastorusingcommoditydisksolutionstoSL85000tap• Tieredstorage,lookingatobjectstorageforcapacityinthenextiteration.• Tableau.RforGenome-WideAssociationStudies(ifthosecountas"BigData")• Hadoop,Cassandra,hive,pig,spark,maven,NoSQLdatabases.• Afilestorededicatedtoresearchdata,abaselevelserviceisfreeofcharge.Researchgroupscanpurchaseadditional

storage.ThefacilitylinkstotheHPCfacility.• Preparinginfrastructure(storage,networks)tosupportfuture'BigData'research.• Galaxywebservice,Hadoop,NoSQLdatabases.• Nothingatpresent.WillbelookingintoHadoopinthenearfuture• Wehavefundedasmallhadoopclusterasatestbed.We'relookingatimagelibrarytechnologies(mainlyformedical

researchprojects)• Hadoop• GPFS,Aridhia,largememorysystems• pbdRsupportedoncluster;sitelicense(viaEngineering)forTecPlotvisualisationsoftwareaswellasseveralopen

sourcedataexplorationandvisualisationtools.• Wehavespecificallydeployedalargesharedmemoryfacilityinsupportofanalysisofpublichealthrecords(Farr

InstitutefacilityaspartofN8HPC)andwehaveadataanalysisclusterdelayedinstitutionally.• nonedirectlyyetbyARC.Otherswithintheuniversityexploredifferenttechnologiessuchashadoop

Page 46: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 46

• Small7nodemaprinstallation.• InstalledRforchem/bioinformatics• SQLservices• Dataanalyticscluster(Hadoop;streaminganalytics);localcloudforbioinformatics

Academicimpact

Q41WhatareyourKeyPerformanceIndicatorstomeasureacademicimpact?

HEILarge/

Specialist Regional GrandTotal

Amountofresearchintermsofresearchgrants 14 4 3 21

Numberofdifferentresearchareasusingthesystems

15 3 2 20

Numberofpost-gradandpost-doctoralusers 6 3 2 11

Numberofprojectsrunonsystems 13 5 3 21

Numberofresearchpaperspublished 14 8 3 25

Numberofthesesproduced 2 2 3 7

Numberofusers 18 5 3 26

Other...... 4 2 1 7

Researchhighlights-highprofilepapers,breakthroughs,newsitems

10 7 2 19

Page 47: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 47

Q42Refereedandconferencepaperspublishedinthelastyear

Q43URLlistingthesepublicationsifapplicable

• http://www.cosmos.damtp.cam.ac.uk/info/cosmos-publications-2014/• http://www.dirac.ac.uk/science_all.html• Separatereportavailableonrequest.• https://pure.strath.ac.uk/portal/en/equipment/uoshpc(a35d23ad-d4fa-481c-b7db-e535add521f7).html• Containedinaseparatereport-availableuponrequest• Seebelowreport

Q44DoesyourorganisationproduceanAnnualReport?Ifso,pleaseenteritsURLbelow

• COSMOSannualreportsarepartoftheDiRACAnnualReportsathttp://www.dirac.ac.uk/• http://www.dirac.ac.uk/science_all.html• http://www.ebi.ac.uk/about/brochures• Differentreportsfordifferentprojectsandservices• http://www.stfc.ac.uk/SCD/resources/PDF/SCD_Science_Highlights_2014.pdf• Inprogress• http://cics.dept.shef.ac.uk/reports/cics-annual-report-2013.pdf• http://www.cf.ac.uk/arcca/news/annualreport2015.html(inpress)• http://www.kcl.ac.uk/newsevents/publications/report.aspx• OurannualreportisaninternaldocumentandpresentedtoourHPCBoard.However,wedoproducepublicreports

usuallytocoincidewithanewsystem,e.g.:https://www.acrc.bris.ac.uk/acrc/HPC_report.pdf• http://www.cam.ac.uk/annual-report• https://www.liv.ac.uk/annual-report/• http://n8hpc.org.uk/n8-hpc-annual-survey-2014-released/

0

0.5

1

1.5

2

2.5

0 50 100 150 200 250 300

Numberofrespondents

Q42.Refereedandconference paperspublished

HEI

Large/Specialist

Regional

Page 48: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 48

• http://www.it.ox.ac.uk/about/reports/it-services-annual-report-20132014

Q45DoesyourorganisationpublishResearchHighlights?IfsocanyouprovideURLsforupto3recenthighlights

• http://www.cosmos.damtp.cam.ac.uk/info/research-highlights-2013-14/• http://www.dirac.ac.uk/science_news.html• Differentfordifferentprojectsandservices• ForSCARF:http://www.scarf.rl.ac.uk/sites/default/files/docs/RAL-TR-2014-017.pdf• https://www.sanger.ac.uk• Yes-toappearonnewwebsitethatiscurrentlyunderconstruction-availableonrequest• http://www.archie-west.ac.uk• http://hpchub.sites.sheffield.ac.uk/research-groups• https://www.nottingham.ac.uk/research/news.aspx• Yes-containedintheannualreport• http://www.ucl.ac.uk/research-it-services/rits-case-studiesmoreindevelopmentatpresent• Yes,buttheseareusuallypublishedbyourPRO,forexample:

http://www.bristol.ac.uk/news/2014/november/hendra-in-bats-and-humans.html• https://www.liv.ac.uk/research/news/• Inamannerofspeaking,againseeN8HPCwebsiteandannualreport.• http://www.sussex.ac.uk/research/

Whousesyourservice?

Q46DoyouprovideHPCservicesbeyondyourimmediateorganisation?

HEILarge/

Specialist Regional GrandTotal

HEIs 3 3 1 7

Other....... 3 3 1 7

DiRAC 2 4

6

GridPP 3 2

5

N8HPC 4 1

5

BiochemistryandLifeSciencesInformatics

1 3

4

ARCHER 3 1

4

FarrInstitute 3 1

4

Page 49: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 49

SES/CFI 2 1

3

JASMIN/CEMS

2

2

CancerResearchUK 1 1

2

GenomicsEngland 1 1

2

HPCWales 1

1 2

NationalOceanographicCentre 1 1

2

ResearchDataFacility 1 1

2

MidPlus 2

2

ARCHIE-WeSt

1 1

DiamondLightSource

1

1

ELIXIR

1

1

HPCMidlands

1 1

HartreeCentre

1

1

MedicalBioinformaticsInitiative

1

1

WellcomeTrustSangerInstitute

1

1

AdministrativeDataResearchCentres

TheGenomeAnalysisCentre

Q47Whatservicesdoyouprovidetothirdparties?

HEI Large/Specialist Regional GrandTotal

Computeandstorage 8 10 3 21

Supportandtraining 3 6 3 12

Dataservices

7 2 9

Page 50: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 50

Networking

7 2 9

Softwaredevelopment 1 4 1 6

Businessdevelopment 1 2 2 5

Other........ 1

1

Q48HowmanyHEIsresearchgroupsuseyourservices?

Q49Publicandthirdsectororganisations

HEI Large/Specialist Regional GrandTotal

NHSand/orDepartmentofHealth

3 6 2 11

Charitablesector

6 1 7

DepartmentforBusiness,InnovationandSkills

1 5

6

EuropeanCommission

2

2

EuropeanSpaceAgency

1 1 2

Other......... 1 1

2

DepartmentofEnergyandClimateChange

0

0.5

1

1.5

2

2.5

3

3.5

0 20 40 60 80 100 120

Numb

erofres

pondents

Q48HowmanyHEIs'researchgroupsuseyourservices?

HEI

Large/Specialist

Regional

Page 51: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 51

Q50Whatpercentageofyourusersarefromindustry?

Q51Ofindustryusers,whatproportionareSMEs?

Q52WhatpercentageofsystemtimeisbeingusedbySMEs?Percentageofsystemcomputeresourceoveratwelvemonthperiod

Q53AreyoutakinganyspecificstepstoincreaseSMEuptake?

• Yes;advertisementandpersonalapproach.• Industryprogramme

0

50

100

150

Q50.Percentageofindustrialusers

Large/specialist

Regional

HEI

0

50

100

150

Q51.PercentageofindustrialuserswhoareSMEs

Large/specialist

Regional

HEI

0

5

10

15

20

25

Q52.PercentageofsystemomeusedbySMEs

Large/specialist

Regional

HEI

Page 52: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 52

• HavealwaysbeenontheforefrontofengagingSMEstouseHPCandcontinuetoworkextensivelyinthisarea.• Yes.ThisistheHartreeCentremission.WehaveaBusinessdevelopmentteamof5peopledevotedtothis.• TheHartreeCentreisdrivingSMEuptakeofHPCandwillprovideaseparateresponsetothissurvey.• No,focusinthesectorismoreonlargerPharma.• InstituteisplanningaBiodataInnovationCentreforupto30SME's• HPCWalesStage1iscurrentlycomingtoaclose,withthedirectionoftravelforitssuccessorprojectyettobe

quantified.WhileSMEengagementwillremainofimportanceitislikely,basedoninitialdiscussionswithWEFOandtheWG,thattheprofilewillbroaden

• Yes,dedicatedbusinessdevelopmentmanagerlookingtoworkwithSMEs• HoldingknowledgeexchangeeventsandexhibitingatindustryeventssuchasNAFEMS.Wehavealsoundertakena

securityauditofourfacilitybyanISOcompliantorganisation.WearealsointheprocessofattemptingtosecureinternalfundsforafulltimeBusinessDevelopmentrole

• Workwithbusinessengagementteams,theAdvancedManufacturingResearchCentreandAdvancedComputingResearchCentre.

• AsapartnerinHPCWales,wearedevelopingourindustrialcontactsthroughthismechanism.• TheCOREinitiativewithImperialCollegebringsthesepeopleinfromtimetotime.• OurstrategyforSMEinvolvementmirrorsourembeddeduseofinstitutionalBEteamswhichwecoordinateinorder

tosupportresearchthatusesHPCtoengagewithindustry.Broadly,ourapproachistotargetsupplychainsoflargercompanies.

• IncreasingengagementwithUniversityconsultingservicestobroadenoutreachofHPCactivity.• LiaisonwithSussexInnovationCentre

Q54Whatsectorsarebeingservedbyyourindustrialusers?

HEI Large/Specialist Regional GrandTotal

Advancedmaterialsandmanufacturing

7 1 3 11

Energyandenvironment 6 1 3 10

Lifesciences 3 4 2 9

Financeandprofessionalservices 3 1 2 6

Defenceandsecurity 4 1 1 6

Transport 3 1 1 5

Creativeindustries 1 1 2 4

Construction 2 1 1 4

Other..........

1 1 2

Page 53: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixF–Servicemanagement:Fullbreakdownofsurveydata 53

Furtherinformation

Q55Pleaseusethisspaceforanyfurtherinformationyouwouldliketoprovide

• Aswiththesystemsurvey,thishasproveddifficulttocompletefromtheperspectiveofadistributedorganisationthatisprovidingHPCservicestobothacademicandthirdpartyorganisations.2.Theabilitytosaveandrestartthesurveywouldhavebeenhelpful.

• SoftwarelicensingstillamajorissueforSME's,bothintermsofprovidingabarriertoaccessandintermsofactingasaconstraintontheextenttowhichtheycanleveragethepowerofHPC.Consequently,thegainswearemakingwithSME'sarenotreflectedinthereport.

• Giventhelengthofthesurveyitwouldhavebeenhelpfultohavea"savenowandcompletelateroption"-plusbeingabletoprintormakecopiesofthecompletedsurveyreportforourownrecords(helpsfutureresponses!)

• Thesurveyquestionsareoftenunclear.

• ThereisactiveindustrialworkgoingonwiththeseparateVEC(VirtualEngineeringCentre)inpartnershipwithHartree.AllindustrialuseofLiverpoolHPCfacilitiesareviacollaboratingacademics.

Page 54: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixG–Hardware:Thesurveyquestions 54

AppendixG–Hardware:Thesurveyquestions

Q1OrganizationnameQ2OrganizationalunitQ3EmailaddressQ4JobtitleQ5SystemnameQ6ExternalIPaddressofFQDNQ7URLforthewebsiteofthesystemoroverallserviceQ8Whatarethetopthreeresearchareasthesystemisusedfor?

Hardwarespecifications

Q9TotalnumberofprocessorcoresinthesystemQ10NumberofcomputenodesQ11NumberofprocessorcorespercomputenodeQ12RAMpercore(Gigabytes)Q13Computenodeprocessorspecificatione.g.IntelIvyBridgeE5-2670v22.5GHzQ14HowmanyGPUequippednodesdoesthesystemhave?Q15HowmanyXeonPhiequippednodesdoesthesystemhave?Q16Howmany"fat"nodesdoesthesystemhave,i.e.>=8GBRAMpercoreQ17DoesthesystemhaveadedicatedVisualizationcapability?PleasedescribebelowifapplicableQ18InterconnectSwitchFabrice.g.QDR/FDRInfiniBand,GigabitEthernet,NUMAlinkQ19Whenwasthesystemcommissioned?Q20Whenwillmaintenanceforthesystemterminate?

Storage

Q21Describethestoragecomponentofthesysteme.g.NetAppCDE5400,PanasasActivStor11,DirectAttachedStorageoncomputenodesQ22TotalusablestorageforHPCusersQ23Whatfilesystem(s)and/orobjectstoresdoyouuseforsharedstorage?Q24DoyousplitsystemstorageintermsofTBbetweenfast,tertiary,archivestorage?Q25Numberofregisteredusers

Performanceandconnectivity

Q26TheoreticalPeakPerformance(Tflop/s)Q27NodetoNodeDataRate(Gbit/s)Q28Averagenodetonodelatency(Microseconds)

Page 55: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixG–Hardware:Thesurveyquestions 55

Q29TypicalCPUloadasa%ofoverallsystemQ30PeakinboundsustaineddatatransferratesQ31PeakoutboundsustaineddatatransferratesQ32IsthebandwidthabovededicatedforHPCserviceuse?Q33Specialconnectivityrequirementse.g.Lightpath,dedicatedcircuit,lowlatency

Softwareandoperatingenvironment

Q34InstalledsoftwareQ35WhatistheprimaryOperatingSystemyouuseoncomputenodes?Q36WhatistheprimaryOperatingSystemyouuseonhead/loginnodes?Q37Whatscheduler(s)doyouuse?Q38DoyouprovideaWebPortaltoyourusers?Ifso,pleasedescribebelowQ39DoyoubackupHPCuserdata?Q40Doyoudohavescheduledmaintenanceandifsohowoften?

Access,authorization,accountingandidentities

Q41ManagementofHPCsystemusersandprojectsQ42Accountingandresourceallocation

Furtherinformation

Q43Pleaseusethisspaceifthereisanyotherinformationyouwouldlikeshareaboutthesystem

Page 56: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixH–Hardware:Summaryofthesurveydata 56

AppendixH–Hardware:Summaryofthesurveydata

Scienceareas-ofthe75systemsthatrespondentsreportedon,keyapplicationareaswereComputationalFluidDynamics(24%)materialsscience(16%)lifesciencesandbioinformatics(37%)andphysics(41%).

Hardware-withthegrowthinprocessorcorecounts,manyinstitutionalsystemswerenowonabar(incorecount,butnotnecessarilyincapacity)withanumberoftheregionalandlarge/specialistfacilities.SeveralHEIsalsohadmorecomputenodesthantheregionalcentresandanumberofthelarge/specialistfacilities.Thiscouldbeexpectedtochangewiththenextroundofhardwarerefresh.39%ofthesystemsinthesurveynowhadGPUequippednodes,althoughinmostcasesthesewere20nodesorlessoutofawholecluster.19%ofthesystemsinthesurveyhadXeonPhiequippednodes,butvirtuallyallofthesewere4orlessnodesoutofacluster.

Fat(highmemory)nodes-thevastmajorityofHEIandregionalsystemsstillonlyhaveahandfulof"fat"nodes,with8GBormoreofmemorypercore.Systemswithhundredsofhighmemorynodesarestillthedomainofthelargeandspecialistfacilities.

Visualization-37%ofrespondents'systemshadadedicatedvisualizationcapability.

Interconnect-48%ofrespondents'systemsusedQDRInfiniband,withagrandtotalof65%ofallsystemsinthesurveybeingconnectedviaInfiniband.23%ofsystemsinthesurveywereconnectedwithGigabitEthernet,with13%operatingat10Gbit/s,and2systemsoperatingat40Gbit/s.

Lifecycle-manyrespondentswerestillrunningoff-maintenancesystemscommissionedasfarbackas2007and2009.8newsystemshadbeencommissionedin2014,andafurther6in2015.26systems(35%oftheUKNeI)wouldfalloffmaintenancein2015and2016.

Storage-institutionsarenowstartingtoinchtowardsthesortofPetabytescalestoragesolutionswhichhadbeenverymuchthedomainoflargeandspecialistfacilities,with5HEIshavingstoragefacilitiesof1PBorabovefortheirscientificcomputingoperation.Lustre(33%),NFS(25%)andGPFS(18%)arestillbyfarthemostcommonfilesystems,althoughgrowthinPanasasinstallationshasseenPanFSriseto14%ofsurveyrespondents'systems.

Userbase-28%ofthesystemsinthesurveyhavelessthan100users,withonly16%havingmorethan1000users.

Performance-aswithcorecounts,institutionalsystemshavestartedtocatchupwithtier1and2facilities,with21outof75(28%)ofsystemsinthesurveyrunningat100Teraflops/sorbetter.However,notallrespondentsprovidedperformancestatsfortheirsystems.

Utilization-51%ofsystemswererunningat75%loadorabove,includingoversubscription,with75%generatingpeakinbounddataratesofbetween1and10Gbit/s.19%ofthesystemsinthesurveyweregeneratingover10Gbit/soutboundtraffic.30%ofsystemshadbeenprovidedwithdedicatedbandwidth,eitherintheirownright,orsharedwithother

Page 57: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixH–Hardware:Summaryofthesurveydata 57

equipment.

Operatingenvironment-59%ofsystemsusedanunencumberedLinuxdistribution(suchasDebianorCentOS)astheirprimarycomputenodeoperatingsystem,with49%alsousinganunencumbereddistributionfortheirloginnodes.Bycontrast,fullysupporteddistributionssuchasRedHatEnterpriseLinux(RHEL)werefoundon24%ofloginnodesand15%ofcomputenodes.OnlyonerespondentwasrunningWindowsHPCServer.

Scheduler-SunGridEngineoritsderivativescontinuetodominatetheschedulersinthesurveyresponses,runningon24%ofsystems.However,SLURMhasnowgrownto16%oftheinstalledbase,with12installationsacrosstheUKNeI.

Backups-32%ofsystemsarenotbackedupatall,versus30%thatarebackedup,and37%wheresomedata(notnecessarilyincludinguserfiles)isbackedup.

Scheduledmaintenance-35%ofsystemsinthesurveydonothavescheduledmaintenance.43%ofsystemshaveeithermonthly,quarterlyorannualscheduledmaintenance.

Managementofusersandprojects-41%ofUKNeIsystemshaveauthenticationlinkedtoinstitutionalsystemssuchasActiveDirectory.However,44%ofsystemshaveatleastsomemanuallycuratedaccounts.32%ofsystemshaveapeerreviewprocessfornewprojects,althoughjust19%haveanequivalentprocessfornewuseraccounts.Only9%ofrespondentsautomaticallydeleteexpireduseraccounts,andjust11%updateuseraccountsautomaticallytotakeaccountofchangessuchasmovingdepartments.13%ofrespondentswereinterestedintriallingJisc'sAssentservice(formerlyProjectMoonshot).

Accountingandresourceallocation-51%ofrespondentsreportedthatprojectsontheirsystemsweregivenaresourceallocation,however23%statedthattheydidnotimposeanyresourcelimitsontheirusers.16%(12institutions)reportedthatusageoftheirsystemswasoftenconstrainedbysoftwarelicenses.

Page 58: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeISurveyReport-2015

ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata

Q8Whatarethetopthreeresearchareasthesystemisusedfor?

TopresearchareasNumberofresponses

AdaptiveSystems 1

AeronauticalEngineering 1

AerospaceEng 1

Astronomy 3

Astrophysics 6

Atomicstructure 2

BigDataandDataAnalytics 2

Biochemistry 1

Bioinformatics 4

BiologicalSciences 4

BiomedicalSciences 2

CFD 18

CancerResearch 1

ChemicalEngineering 1

Chemistry 8

CivilandEnvironmentalEngineering 2

Climate/OceanModelling 4

Page 59: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 59

ComputationalChemistry 4

ComputationalMedicine 1

ComputationalNeuroscience 1

ComputationalPhysics 1

ComputerAidedFormulation 2

CondensedMatter 2

Cosmology 5

Cryo-electronmicroscopy 1

EarthScience 4

Economics 1

ElectronicSystemDesign 1

EnergyEfficientComputing 1

EnergyEfficientTransport 1

Engineering 7

EnvironmentalGenomics 1

Exoplanets 1

FEA 2

FundamentalPhysics 1

GaitAnalysis 1

GalaxyFormation 2

Genomics 4

GeographicalSciences 1

Healthcare 2

HighEnergyPhysics 1

Hydrology 1

Informatics 2

Page 60: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 60

LifeSciences 6

MHD 2

MachineLearning 1

MaterialsScience 12

Mathematics 5

MechanicalEngineering 1

MedicalInformatics 1

Microbialbioinformatics 1

MolecularDynamics 3

NaturalLanguage 1

Nextgenerationsequencing 1

Optoelectronics 1

ParticlePhysics 5

Physics 6

PlasmaPhysics 3

Pyschology 1

QCD 1

RNAsequencing 1

Satelliteimages 1

Science 1

SemiconductorDeviceModelling 2

SoftMatterPhysics 1

Speechandimageprocessing 1

Weather/climate 1

Page 61: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 61

Hardwarespecifications

Q9Totalnumberofprocessorcoresinthesystem

Q10Numberofcomputenodes

Q11Numberofprocessorcorespercomputenode

0

50000

100000

150000

Q9Totalnumberofprocessorcoresinthesystem

Large/Specialist

Regional

HEI

0

1000

2000

3000

4000

5000

6000

7000

Q10Numberofcomputenodes

Large/Specialist

Regional

HEI

0500100015002000250030003500

Q11Numberofprocessorcorespercomputenode

Large/Specialist

Regional

HEI

Page 62: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 62

Q12RAMpercore(Gigabytes)

Q13Computenodeprocessorspecificatione.g.IntelIvyBridgeE5-2670v22.5GHz

• 16xE55302.40GHz32xX56502.67GHz2xAMD61742.2GHz30xE5-2650v22.6GHz32xAMD63782.4GHz• 220*E5520,156*E5-2650v2• AMDOpteron62762.3GHz• AMDOpteron63762.3GHz• AMDOpteronProcessor63782.4GHz• E5-2580v2• E5-4620v22.6GHz• E54622.8GHz,E5-26702.6GHz• IBMPower• [email protected]• Intel• [email protected]• IntelE52650V2• IntelE5-26702.60GHz• IntelE5-4650L2.6GHz8coreXeonprocessor• IntelE5620• IntelHaswellE5-2640v32.6GHz• IntelIvyBridgeE5-26502.6GHz• IntelIvyBridgeE5-2650v22.6GHz• IntelSandyBridgeE5-26402.50GHz• IntelSandyBridgeE5-2650• IntelSandyBridgeE5-2660• IntelSandyBridgeE5-26702.6GHz,IntelWestmereX56602.8GHz,IntelHaswellE5-2680V32.5GHz• [email protected]• IntelSandybridgeE5-16503.2GHz• IntelSandybridgeE5-26502.0GHz• IntelSandybridgeE5-26702.6GHz• IntelWestmereE5-2697X56502.67GHz;IntelSandyBridgeE5-26902.9GHz;IntelSandyBridgeE5-26702.6GHz;• IntelWestmereX5650• IntelXeonE5-26602.2GHz• IntelXeonE5-2695v2• IntelXeonE55302.40GHz• IntelXeonX56602.80GHz

0

20

40

60

80

100

120

140

Q12RAMpercore

Large/Specialist

Regional

HEI

Page 63: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 63

• Intel(R)Xeon(R)[email protected]• Intel(R)Xeon(R)[email protected]• Intel(R)Xeon(R)[email protected]• Intel(R)Xeon(R)[email protected](SandyBridge)• Intel(R)Xeon(R)[email protected]• Intel(R)Xeon(R)[email protected]• Intel(R)Xeon(R)[email protected]• IvyBridgeUoB:E5-4620v22.6Ghz,E7-8857v23.0Ghz,E7-8880v22.5Ghzothersites:E5-4610v22.3Ghz,E7-8850-v2

2.3Ghz• Ivybridge-v2E5-2650v22.6GHz• MixofWoodcrest,Nehalem,IvyBridge(variousmodelsthereof)• N/A• NA• [email protected],[email protected]• SCARFhasmanygenerationsofprocessorasthemodelistoupgradeityearly• SandyBridge,Westmere,Haswell,Opteron• [email protected]• Westmere2.8/2.95GHz• XeonE5-4620v2(20MCache,2.60GHz)• Xeon(R)CPUE5-26702.60GHz(3328cores)for"regular"computenode• latestintelgeneration• varies• variousgenerations• variousgenerationslatestinstall2015• westmere/ivybridge

Q14HowmanyGPUequippednodesdoesthesystemhave?

0

50

100

150

Q14.HowmanyGPUequippednodesdoesthesystemhave?

Large/Specialist

Regional

HEI

Page 64: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 64

Q15HowmanyXeonPhiequippednodesdoesthesystemhave?

Q16Howmany"fat"nodesdoesthesystemhave,i.e.>=8GBRAMpercore

Q17DoesthesystemhaveadedicatedVisualizationcapability?Pleasedescribebelowifapplicable

0

10

20

30

40

50

Q15.HowmanyXeonPhiequippednodesdoesthesystemhave?

Large/Specialist

Regional

HEI

0100200300400500600700

Q16Howmany"fat"nodesdoesthesystemhave?

Large/Specialist

Regional

HEI

No63%

Yes37%

Q17.DoesthesystemhaveadedicatedVisualizaooncapability?

Page 65: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 65

Q18InterconnectSwitchFabrice.g.QDR/FDRInfiniBand,GigabitEthernet,NUMAlink

Q19Whenwasthesystemcommissioned?

Q20Whenwillmaintenanceforthesystemterminate?

10GbE13%

1GbE7%

40GbE3%

BG/Q5DTorus2%

CrayAries1%FDR

Infiniband10%

IBM3%Infiniband

7%

QDRInfiniband

48%

SGINUMAlink

6%

Q18.Interconnect

2 2

6 6

13

8 8

6

0

2

4

6

8

10

12

14

2007 2009 2010 2011 2012 2013 2014 2015

Q19.Datecommissioned

21

13 13

7

43

4

0

2

4

6

8

10

12

14

2013 2014 2015 2016 2017 2018 2019 2020

Q20.Endofmaintenance

Page 66: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 66

Storage

Q21Describethestoragecomponentofthesysteme.g.NetAppCDE5400,PanasasActivStor11,DirectAttachedStorageoncomputenodes

• 150TBLustre• 22TBScratchusingSGIStorageDirectlyAttached1xPanasasActivStor11• 250TBlustrescratch0.51PTIsilonresilientstorage• 2xSL8500taperobots(1usedbyTier1)with10000slotseachwithatotalnearlinecapabilityof160PB(whenfully

populated)TapeServers,OracleDatabases,DataTransferNodessupportingCASTORandDMFfrontendedbya700TBdiskcacheMonitoringSystems

• 7shelvesofPanasasActiveStor11• 800TBytesLustreDellPowerVault200TBytesNFSPowerVault• 8xHPP2000G3SASdisktrays• BlockdeviceaccessibleviaVMinuserconfiguredway• Cloud:Isilon&SAN,Cluster:GPFS• DDN• DDN9550,Panasas14s• DDNSFA10K• DDNSFA12K-40• DellMDstorage.• DellMD3000• Dellcommodity• DirectAttachonServer• DirectAttachedStorage• DirectAttachedStorageoncomputenodes• DirectAttachedStorageonstoragenodes• DirectAttachedstoragethroughheadnode• Directattached14PBofcommoditystorageusing400diskservers.SL85000taperobotwith>24driveswith14PB

managedbyCASTOR• Directattachedstorage• Directattachedstorageonnodes(0.5TBpernode)DDNSFA10K/SFA12KHPSL4540• EMCIsilon• Eachsitehas~400TBGPFSattachedtohypervisornodesforVMimagesandworkingdata.6.9PbofCEPHobjectstore

acrossthesitesisbeingcommissionedstill• Fast-PanasasActiveStor12Tertiary-HNASAMS2500&HUS150• Fraunhofer/BeeGFSparallelfilesystemoverIBusingDellMD3460• GPFS• GPFSBaseduponDDNdiscsviaInfiniband• GPFSBaseduponDDNdiscsviaInfiniband.GPFSbaseduponIBMGSS24• GPFSFPObaseduponIBMGSS24• GPFSacrossclusterwith250GBlocalstoragepernode• GPFSbaseduponIBMGSS24andGPFSbaseduponflashmemorydirectlyconnectedtoI/ONodes• IBMDS3512,DS3524andIBMDCS3700

Page 67: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 67

• IBMDS5300• IBMGSS24+GSS26• IBMgeneralparallelfilesystem(GPFS),givingaround~110TBofusablestoragedividedamongtheusersandsplit

betweenpermanentstorageandtemporary/scratchspace.• LSI• LSISAS• LongTermNAS:HPSL4540basedNFSstoragetotalling696TBusable,splitacrosstwositesfordisasterrecoveryHigh-

speedscratch:IntelEnterpriseEditionLustreonSupermicrobasedhardwareprovidedbyBoston/BIOS-IT,2filesystemseachwith512TBusable

• Lustre120TB• LustreNFS• LustreParallelfilesystem,NFShomefilesystem,Nexenta• LustrescratchpartitionandbackedupNFShome• Netapp176TBusable(fordataandhomeareas)andlustre260TB(fastdataareawith90daylifetimepolicy)• PanasFSandFhGFS• PanasasActivStor11• PanasasActivStor11GPFS• PanasasActivStor12• PanasasActivStor14• PanasasActiveStor• PanasasActiveStor12• PanasasActiveStor14• PanasasActiveStor8(beingupgradedto16thisyear)forgeneraluseMultipleNFSdiskstoragenodesforHighEnergy

Physics• RAID5arraywithinheadnode• SGICXFS• SGIIS5000• SGIInfiniteStorage5000RAIDArrayIS5000dualcontroller2U24-bay2.5”RAIDarraycontaining:o20x3000GB7.2K

rpm2.5”6Gb/sSASHDD• StoragearraycomprisingDellMD3200andMD1200unitswithRAID6arrays• StorageisNFSmountedfromasinglestoragenode.• Sun/OracleSnowbirdsystemmadeupof8J4440arrays• TwoNetAppE2600• VariousDellMDstorage.

Page 68: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 68

Q22TotalusablestorageforHPCusers

Q23Whatfilesystem(s)and/orobjectstoresdoyouuseforsharedstorage?

Q24DoyousplitsystemstorageintermsofTBbetweenfast,tertiary,archivestorage?

0

10000

20000

30000

TB

Q22.Totalusablestorageforusers

Large/Specialist

Regional

HEI

1

18

25

33

14

0

5

10

15

20

25

30

35

Ceph GPFS Lustre NFS PanFS

Q23.Filesystem/objectstore

38

21

9

Fast Ter|ary Archive0

5

10

15

20

25

30

35

40

Q24.Storagesplit

Page 69: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 69

Q25Numberofregisteredusers

Performanceandconnectivity

Q26TheoreticalPeakPerformance(Tflop/s)

Q27NodetoNodeDataRate(Gbit/s)

0-10028%

100-20018%200-500

22%500-750

7%

750-10009%

1000-20008%

2000-50007%

Over100001%

Q25.Numberofregisteredusers

0

500

1000

1500

2000

2500

3000

Tflop/s

Q26.TheoreocalPeakPerformance

Large/Specialist

Regional

HEI

0

10

20

30

40

50

60

Gbit/s

Q27.Nodetonodedatarate

Large/Specialist

Regional

HEI

Page 70: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 70

Q28Averagenodetonodelatency(Microseconds)

Q29TypicalCPUloadasa%ofoverallsystem

Q30Peakinboundsustaineddatatransferrates

0

20

40

60

80

100

120

Milliseconds

Q28.Averagenodetonodelatency

Large/Specialist

Regional

HEI

Commissioning5%

<=25%6%

26%to50%6%

51%to75%32%

>75%45%

Oversubscribed6%

Q29.TypicalCPUloadasa%ofoverallsystem

100Mbit/sto1Gbit/s

2%

1Gbit/sto10Gbit/s75%

Other4%

Unsure15%

Q30.Peakinbounddatarates

Page 71: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 71

Q31Peakoutboundsustaineddatatransferrates.Inmostcasesthiswillbetheconnectionspeedoftheorganizationoverall,butifbandwidthisspecificallydedicatedforHPCpleaseindicatebelow

Q32IsthebandwidthabovededicatedforHPCserviceuse?

Q33Specialconnectivityrequirementse.g.Lightpath,dedicatedcircuit,lowlatency

• Replicatedstoragebetweensitesmayinthefuturerequirededicatedconnectivity.Wecurrentlyareprovisioningipsectunnelsbetweensitesforsecurereplicationofdata.

• Canconfigurelightpathordedicatedconnectionsasrequired.• 10GbexternallinktoJanet,40GBinternalbackplane.• OPNtoCERNat10Gbits/s.shared40Gbits/stoJanet.• TheNSCCSmachineisalargesharedmemorymachinewith512Coresand4TBofmemory.Thisarchitecturesuitsthe

computationalchemistryapplications.• OPNstoMetOfficeExeter,ARCHER,LeedsUniversity,SpaceApplicationsCatapult.• Lightpathanddedicatednetworkscanbeconfiguredonrequest.• AfunctionofApplicationSector-directconnectivitytoNGSsystemsdemandspeakperformance.• Noneatpresent,butNGSaccessdictatesanorderofmagnitudeincreaseinconnectivityperformance.

<1Gbit/s9%

1Gbit/sto10Gbit/s55%

>10Gbit/s19%

Other4%

Unsure13%

Q31.Peakoutbounddatarates

Yes,singlesystem18%

Yes,mul|plesystems22%

No50%

Other4%

Unsure6%

Q32.Dedicatedbandwidth

Page 72: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 72

Softwareandoperatingenvironment

Q34Installedsoftware

Theresultsfromthisquestionwereinconclusiveduetoabuginthesurveytool.

Q35WhatistheprimaryOperatingSystemyouuseoncomputenodes?

Q36WhatistheprimaryOperatingSystemyouuseonhead/loginnodes?

11 13

44

16

05

101520253035404550

Fullysupported1stpartyLinuxdistribu|on

HPCVendorre-spinofexisitngLinuxdistribu|on

UnencumberedLinuxdistribu|on

WindowsHPCServer

Other

Q35.PrimarycomputenodeOS

18

6

37

1

7

0

5

10

15

20

25

30

35

40

Fullysupported1stpartyLinuxdistribu|on

HPCVendorre-spinofexisitngLinuxdistribu|on

UnencumberedLinuxdistribu|on

WindowsHPCServer

Other

Q36.PrimaryloginnodeOS

Page 73: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 73

Q37Whatscheduler(s)doyouuse?

Q38DoyouprovideaWebPortaltoyourusers?Ifso,pleasedescribebelow

• Horizoniscurrentlyonlywayformostuserstoaccess.APIaccessprovidedviacontrollers.• Onrequest• Yes,SAFE:https://www.archer.ac.uk/safeAccountManagementProjectManagementReportingHelpdeskIncident

ManagementSystemConfiguration• http://www.cosmos.damtp.cam.ac.uk/• Yes-OpenStackHorizon• AVFisaphysicalfacilitywhereuserscomeandexploretheirdataandinteractin3D.AVFcanalsoofferremote

visualisationaswell.• CertificateWizardApplicationhttp://www.ngs.ac.uk/ukca/certificates/certwizard• Yes,needsauthenticationtousehttps://goc.egi.eu/portal/• SCARFisaccessibleviathePlatformApplicationPortal.Thisallowsuserstouploadanddownloadfiles,submit

computationaljobsanddesignworkflows.• WebaccessviathePlatformApplicationPortal• Userprovisionedcloudwebportalavailable.• ThewebportaltotheaccountingdataisprovidedbyCESGAhttp://accounting.egi.eu/egi.php• Yes,basedonFujitsu'sSynfiniWaymiddleware;seehttps://portal.hpcwales.co.uk• Yes,foraccessdetailsandusageinformation.Notforjobsubmission.• http://www.cfi.ses.ac.uk/cfi/iridis/• Yes-Galaxy(LifeSciences),WebMO(Chemistry),GridChem(Chemistry)andweareintheprocessofevaluatingboth

Altair'sComputeManagerandOpen-SourceCylc(asamulti-purposeweb-basedinterface).• Forgaussianonly• EvaluatingAltairComputeManager,notinproductionuse• Oraclesecureglobaldesktophttp://www.oracle.com/us/technologies/virtualization/secure-global-

desktop/overview/index.html• https://maxwell.abdn.ac.uk-providedbyAlces• StandardBrightClusterManagerwithmodificationsbyClusterVisionforVisualisationservice

13

6

14

912

18

8

0

5

10

15

20

Q37.Scheduler

Page 74: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 74

Q39DoyoubackupHPCuserdata?

Q40Doyoudohavescheduledmaintenanceandifsohowoften?

Access,authorization,accountingandidentities

Q41ManagementofHPCsystemusersandprojects

Yes30%

Somedata37%Onlyhome

dirs1%

No32%

Q39.Backups

Monthly22%

Quarterly10%

Annually11%

Other22%

No35%

Q40.Scheduledmaintenance

8

31

10

21

7 74

10

33

4

14

24

05101520253035

Q41.Managementofusers&projects

Page 75: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 75

Q42Accountingandresourceallocation

Furtherinformation

Q43Pleaseusethisspaceifthereisanyotherinformationyouwouldlikeshareaboutthesystem

• CLIMBisadifferentapproachcomparedtotraditionalHPCsystems,somanyofthequestionsdon’tquitefit!StoragecephnodesalsorunRHEL7.1

• TheComputingInfrastructureforScience(CiS)groupinNBIParnershipLtdmanagestheHPCandenterprisestorageforfourInstitutesonasharedcampusnetworkinNorwich:TGAC,JIC,IFRandTSL.ThelargestproportionofHPCandstorageutilisationisfromTGAC.

• TheAVFisaphysicalfacilitywithhighperformance3Dvisualisationcapabilityattachedtohighmemorymachinesfordataanalysis.Theroomisusedcollaborativelywithresearchersexploringtheirdatatogetherandgainingnewinsights.Thehighmemorynodesarecriticalsuchthatlargedatasetscanbemanipulatedinrealtime.Remotevisualisationisalsoofferedtoresearchers

• PerformanceismeasuredbytheHEPSPECratherthaninTflopsasthisismorerelevanttotheworkloads.http://w3.hepix.org/benchmarks/doku.php?id=homepageTheperformanceoftheTier1is~120kHEPSPEC06

• TheAtlasDatastoreisahierarchicalstoragesystemprovidingarchive,backups,datacurationandworkingrepositories.

• TheUKe-ScienceCertificateAuthorityprovidesasecurityinfrastructureforpeopleandsystemswhichisacceptedgloballythroughtheIGTF.TheIGTFistheinteroperableglobaltrustfederation.http://www.igtf.net

• GOCDBistheofficialrepositoryforstoringandpresentingEGItopologyandresourcesinformation.TheGOCDBdataconsistsmainlyof:ParticipatingNationalGridInitiatives(NGI)GridSitesprovidingresourcestotheinfrastructureResourcesandservices,includingmaintenanceplansfortheseresourcesParticipatingpeople,andtheirroleswithinEGIoperations

38

12

2523

17

13

0

5

10

15

20

25

30

35

40

Projectsaregivenaresource

alloca|on

Usageiso}enconstrainedby

so}warelicenses

Useraccountsaregivenaresourcealloca|on

Weac|velymonitorusageagainstresource

alloca|on

Wedonotimposeresource

limits

Other

Q42.Accounong&resourceallocaoon

Page 76: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 76

• SCARFismanagedaspartofaportfolioofservicesthatSCDofferstotheUKandinternationalsciencecommunities.• NSCCSispartofancollectionofservicesthatSCDprovidestotheUKacademiccommunityOverthelastyearNSCCS

hasbeenusedby54researchgroupsfrom24institutionsresultinginover60publications.• Data-intensivecomputingJASMINprovidestheUKandEuropeanNERCfundedenvironmentalsciencecommunities

withanefficientdataanalysisenvironment.Manydatasets,particularlymodeldata,aretoobigtobeeasilyshippedaround:JASMINenablesscientiststobringtheirprocessingtothedata.FlexibledataaccessJASMINprovidesnewwaysforscientiststocollaborateinself-managinggroupworkspaces,enablingmodelsandalgorithmstobeevaluatedalongsidecuratedarchivedata,andfordatatobesharedandevaluatedbeforebeingdepositedinthepermanentarchive.

• APELisanaccountingtoolthatcollectsaccountingdatafromsitesparticipatingintheEGIandWLCGinfrastructuresaswellasfromsitesbelongingtootherGridorganisationsthatarecollaboratingwithEGI,includingOSG,NorduGridandINFN.TheaccountinginformationisgatheredfromdifferentsensorsintoacentralaccountingdatabasewhereitisprocessedtogeneratestatisticalsummariesthatareavailablethroughtheEGI/WLCGAccountingPortal.APELcollectsdatafrom~300institutionsat~3Mrecordsperdaytotalling~400GBofdata.Thisiskeptfor18monthsafterwhichsummarydataisheld.StatisticsareavailableforviewindifferentdetailbyUsers,VOManagers,SiteAdministratorsandanonymoususersaccordingtowelldefinedaccessrights.

• TheaccountinginformationisgatheredfromdifferentsensorsintoacentralaccountingdatabasewhereitisprocessedtogeneratestatisticalsummariesthatareavailablethroughtheEGI/WLCGAccountingPortal.TheAPELsystemreceives~3MrecordsperdayintoaMySQLdatabasefrom314sites.IndividualJobrecordsarekeptfor18months(~400GB).After18monthsthejobrecordsaresummarisedandkeptindefinitely.StatisticsareavailableforviewindifferentdetailbyUsers,VOManagers,SiteAdministratorsandanonymoususersaccordingtowelldefinedaccessrights.

• Backupisfordisasterrecoverypurposesonly• Acommenttopointoutthedifficultyinenteringdatafordistributedsystems-particularlytoughinourcasee.g.,Q19

-couldnotworkouthowtoentermultipledatesforamulti-phaseimplementation-2011and2013.Onethingtoredlineanon-acceptableresponse,butnoindicationofwhatconstitutesavalidresponse.Wouldhavebeenusefultohavehadasaveandrestartcapabilityforaquestionnaireofthissize.Curtailingtheinputonsomeofthelinesshouldbeaccompaniedbyamaximumfiledlength.

• Althoughlistedasasinglesystem,therearethreepartitions-theprimarySandyBridgepartitionisthemainparallelMPIservice,theWestmereisforserialorbatchjobs,andthenewHaswellservicewillhaveamixedworkload(primarilyserialbutwillalsobeavailableforMPI-basedjobs).ThislatterpartitionwasinstalledinJanuary2015andthemaintenancewillexpireinJan2018(butunfortunatelythemaintenancequestiononlypermittedasingledateentry!).

• Heterogeneoussystem,somultiplehardwaretypesandsupportcontractspanning2010todate.Surveydoesn'treallyaccommodatethistypeofcluster.TheclusterisbothourlocalHPCfacilityandHEP/GridPPnode.Fundingisahybridmodel-acentralUniversitycontributionandresearchgrant-fundedcontributions

• Researchershavetoregisteraproject,whichmaybeunfunded,andusershavetobeassociatedwithaproject.Thisenableslinkingresearchstudentstotheirsupervisorwhichisveryusefulincaseofproblems.ItalsoenablesustoidentifytherangeofusersanddisciplineswhichisessentialtomakingthecasetotheUniversityforongoingfunding.

• WeareabouttoundergoasoftwareupgradethatwillratherchangetheOSandapplicationavailability(forexamplewearemovingtoRHELfromSL).Commissiondateisinaccurateasthissystemhasbeeninuseforalongtime.Itisalsonotpossibletogiveamaintenancecutoffbecausethatvariesfromalreadyexpiredtobrandnewequipment.

Page 77: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixI–Hardware:Fullbreakdownofsurveydata 77

• Iskippedthequestionsoncommissioningandmaintenancedates,becauseweupgradeonanannualcycle,andoncetheoriginalsupportagreementexpiresweoftenrolltheHPCnodeintoourcampusmaintenanceagreementwithHP.

• WealsohaveanactiveCondorsystemwithcirca20activeusers,runningacross750campusPCs,thattypicallyaccommodates20,000coredaysofcomputingpermonthviaapproximately60,000jobs(permonth).ThisrelievespressureofftheHPCsystemforhighvolume,highthroughoutjobs.

• Thesystemisheterogeneousandbasedonacontributionmodel.Centralseedfundingboughtinfrastructure.Researchgroupscontributefundstobuycomputenodes.Procurementstakeplacetwoorthreetimesayear.

• TheUniversityfundsageneralHPCservicewhichisaugmentedbytheadditionofresearchgrantorschoolfundedcomputeandstorage.NodesandStoragearepurchasedwith5yearswarranty,afterwhichtheyareconsideredEndofLife.WeuseBrightforClusterprovisioningandmanagement.

Page 78: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 78

AppendixJ–Hardware:Summaryofthesurveydata

Table:ThehardwaresystemsaregivenintheTablebelowwhichissplittointo3sections,eachrepresentingalayerintheBranscombPyramid.Section1liststheLargeandSpecialistSystems.Section2givestheRegionalSystems.Section3liststhefoundationallayer,theHEIsector.

1.LargeandSpecialistServices

Organizationname Systemname

Whatarethetopthreeresearchareasthesystemisusedfor?

Totalnumberofprocessorcoresinthesystem

TotalusablestorageforHPCusers(TB)

Numberofregisteredusers

TheoreticalPeakPerformance(Tflop/s)

CloudInfrastructureforMicrobialBioinformatics-CLIMB-Birmingham,Cardiff,Swansea,Cardiff CLIMB

ResearchinhowmicrobialbioinformaticiansusecloudMicrobialbioinformatics(assembly,analysis)Buildinganacademiccloud 3,864

0-100 74

DiRAC@DurhamUniversity

DiRAC-1@Virgo(COSMA4)

CosmologyGalaxyFormation 7,072 1,100 100-200 32

DiRAC@DurhamUniversity

DiRAC-2@DataCentric(COSMA5)

CosmologyMHDGalaxyFormation 6,720 2,500 100-200 140

Page 79: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 79

DiRAC@EPCC DIRACBG/QQCD,SoftMatterPhysics 98,304 1,000 200-500 1,258

DiRAC@UniversityofCambridge(DAMTP) COSMOS

Cosmology,Astrophysics,Exoplanets 3,284 306 200-500 63

DiRAC@UniversityofCambridge(HPCS) Darwin

LifeSciences.Atomicstructure.ComputationalFluidDynamics. 9,600 2,847

750-1,000 200

DiRAC@UniversityofLeicester Complexity

AstrophysicsParticlephysics 4,352 710 100-200 91

EMBL-EBI-EuropeanBioinformaticsInstitute

EmbassyCloud Lifescienceresearch 31,000 3,200 200-500

eMedLab-TheCrick,UCL,QMUL,LSHTM,EBI,TheSanger eMedLab

Biomedicalresearch(andanyassociatedareas).NextgenerationsequencingRNAsequencingCryo-electronmicroscopy 6,048 4,800 0-100

UVRI/MRCMedicalInformaticsCentre UMIC

MedicalInformatics(notyetinoperation) 2,048 1,720 0-100 19

NorwichBioscienceInstitutes(TGAC,JIC,IFR,TSL)

Bioinformatics,mathematicalmodelling. 9,000 4,000

750-1,000

STFCHartreeCentre BlueJoule

Modelling&Simulation(CFD,Materials,andComputerAidedFormulation) 98,000 6000 200-500 1,200

Page 80: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 80

STFCHartreeCentre BGAS

Modelling&Simulation(CFDandMaterials) 32,000 1000 0-100 450

STFCHartreeCentre BlueWonder

Modelling&Simulation(CFD,Materials,andComputerAidedFormulation) 24,000 9000

750-1,000 200

STFCHartreeCentre EECR/FPGA

EnergyEfficientComputing 2,000 200 0-100

STFCHartreeCentre BigData

BigDataandDataAnalytics 1,184 1000 0-100

STFCScientificComputingDivision SCARF

ComputationalChemistryPlasmaPhysics,ProcessingSatelliteimagesSupportofISIS,CLF,RAPSP,DLSusercommunities 7,000 320 500-750 165

STFCScientificComputingDivision JASMIN

ClimateScience,EarthObservation,environmentalgenomics 4,500 25

Over10,000

Page 81: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 81

STFCScientificComputingDivision

NationalServiceforComputationalChemistrySoftware

TheEPSRCUKNationalServiceforComputationalChemistrySoftware(NSCCS)providesaccesstosoftware,specialistconsultation,computingresourcesandsoftwaretrainingtosupportUKacademicsworkingacrossallfieldsofchemistry.NSCCSsupports127researchgroupsfromdisciplinesincludingChemistry,MaterialsSciences,Physics,EarthScienceandEngineering,Astronomy,Biochemistry,BiologicalSciences,BiomedicalSciences,LifeSciences,CivilandEnvironmentalEngineeringandChemicalEngineering 512 32 200-500

STFCScientificComputingDivision

AtlasVisualisationFacility(AVF)

MaterialsciencetomographyandPlasmaPhysics 112

0-100

STFCScientificComputingDivision

AtlasDatastore

ParticlePhysics,LifeSciences,MaterialsSciencepluseverythingelse

25,000 100-200

STFCScientificComputingDivision

UKe-ScienceCertficationAuthority

SupportsallUKresearch.MajorusersParticlePhysics

750-1,000

STFCScientificComputingDivision GOCDB

Supportsmulti-nationale-infrastructures

2,000-5,000

Page 82: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 82

STFCScientificComputingDivision GridPPTier1

ParticlePhysicsExpandingtosupportotherdisciplines 10,000 28,000

1,000-2,000

STFCScientificComputingDivision APEL

APELisacentralrepositoryforaccounting/usagedata.Itsupportslocal,gridandcloudcomputing.

STFCScientificComputingDivision APEL

APELisanaccountingtoolthatcollectsaccountingdatafromsitesparticipatingintheEGIandWLCGinfrastructuresaswellasfromsitesbelongingtootherGridorganisationsthatarecollaboratingwithEGI,includingOSG,NorduGridandINFN.APELcurrentlysupportslocal,gridandclouddata

200-500

TheInstituteofCancerResearch Multiple

Processingofsequencing,massspecandimagingdata. 1,600 2,500 0-100

EPCC ARCHER

MaterialsScience,Climate/OceanModelling,ComputationalFluidDynamics 118,080 4,608

1,000-2,000 2,550

EPCC Indy Industry/SMEUse 1,536 175 100-200

EPCC UltraHealthcare,Bioinformatics 512 253 0-100

Page 83: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 83

EPCC DIR Dataanalytics 240 2,250 0-100

EPCC UK-RDF

Climate/oceanmodelling,ComputationalFluidDynamics,MaterialsScience

23,0001,000-2,000

FarrNorth,HealtheResearchCentre

HeRCSafeHaven

HealthcareBio-healthInformaticsMachineLearning 256

0-100

WellcomeTrustSangerInstitute

SangerHPCResources Genomics 16,946 9,900

1,000-2,000

LARGE&SPECIALISTTOTALS 499,770 135,446 30,550 6,441

2.RegionalSystems

Organizationname Systemname

Whatarethetopthreeresearchareasthesystemisusedfor?

Totalnumberofprocessorcoresinthe

system

TotalusablestorageforHPC

users(TB)

Numberof

registeredusers

TheoreticalPeak

Performance(Tflop/s)

HighPerformanceComputing(HPC)Wales

Various(distributedsystem)

AdvancedMaterials&Manufacturing,LifeSciencesandEnergy&Environment 16,816 702

2,000-5,000 319

HPCMidlands Hera

AdvancedMaterialsEnergyEfficientTransport 3,008 120 100-200 48

N8HPC Polaris

5,312 175 200-500 138

Page 84: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 84

ARCHIE-WeSt ARCHIEMoleculardynamics,CFD,PlasmaPhysics 3,920 148 200-500 38

SES/CFI IRIDIS3

Chemistryresearch,EngineeringresearchandBiologyresearch 12,000 110 200-500 106

REGIONALTOTALS 41,056 1,255 49,660 649

3.HEISystems

Organizationname Systemname

Whatarethetopthreeresearchareasthesystemisusedfor?

Totalnumberofprocessorcoresinthe

system

TotalusablestorageforHPC

users(TB)

Numberof

registeredusers

TheoreticalPeak

Performance(Tflop/s)

CardiffUniversity Raven

EPSRC(materials,chemistry,engineering),BBSRC(genomics)andNERC(earthsciences) 4,352 275 500-750 110

CranfieldUniversity Astral CFDFEA 1,280 34 200-500 20

DurhamUniversity Hamilton

CondensedMatterMolecularDynamicsFluidDynamics 5,600 350 200-500 75

ImperialCollegeLondon ax3 Genomics 1,300 1,500 0-100

ImperialCollegeLondon cx1

21,558 2,000750-1,000

ImperialCollegeLondon cx2

7,000 500 0-100 60

Page 85: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 85

King'sCollegeLondon ADA

ComputationalPhysics,Mathematics,Informatics 1,624 87 100-200 85

LancasterUniversity

HEC(HighEndCluster)

HighEnergyPhysicsCondensedMatterTheoryCFD 4,784 1,530 200-500

LoughboroughUniversity Hydra

2,460 100 100-200

QueensUniversityofBelfast DellCluster

Chemistry,Physics,CancerResearch 984 40 0-100

QueensUniversityofBelfast

WindowsCluster

biology,speechandimageprocessing,CFD 256 30 0-100

TheUniversityofBirmingham

bluebear.bham.ac.uk

MathematicsCivilEngineeringPyschology 1,632 150 500-750 21

TheUniversityofNottingham Minerva

Chemistry,Engineering,Physics 2,752 180 200-500 55

TheUniversityofSheffield iceberg1

AeronauticalEngineeringandmodellingofturbulentfluidsComputationalMedicineBioinformatics 3,440 40

1,000-2,000 112

UniversityCollegeLondon Legion

Chemistry,Physics,BiologicalSciences(accordingtoREFCategories) 7,816 356 500-750 115

UniversityofAberdeen

maxwell.abdn.ac.uk

LifeSciences-GenomicsCFD-EngineeringMatlab 600 56 100-200

Page 86: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 86

UniversityofBath

Balena/Aquila

ChemistryPhysicsMechanicalEngineering 3,072 220 0-100 64

UniversityofBristol BlueCrystal

Chemistry,AerospaceEng,GeographicalSciences 9,000 740

750-1,000 240

UniversityofCambridge Wilkes

ComputationalFluidDynamics.Atomicstructure. 1,536

0-100 256

UniversityofEdinburgh Eddie

Physics,Informatics,Engineering 3,248 281

1,000-2,000 28

UniversityofExeter

AstrophysicsWeather/climateHydrology 2,184 73 100-200 25

UniversityofGlasgow Conan

SemiconductorDeviceModelling 1,360 40 0-100

UniversityofGlasgow Cnoc

ElectronicSystemDesign 320 10 0-100

UniversityofGlasgow Dusty

ComputationalFluidDynamics 188 4 0-100

UniversityofGlasgow Miffy

SemiconductorDeviceModellingComputationalFluidDynamicsOptoelectronics 1,256 22 0-100 15

UniversityofLeeds Arc1

CFD,Astrophysics,climatescience 4,128 117 500-750 31

UniversityofLeeds Arc2

3,040 175 200-500 316

Page 87: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 87

UniversityofLeicester ALICE

Astrophysics(nowlargelymigratedtoDiRAC)EarthObservationScienceEngineeringEconomics 3,972 1,644 200-500 101

UniversityofLiverpool chadwick

MaterialsmodellingComputationalFluidDynamicsGaitAnalysis 3,180 132 100-200 23

UniversityofManchester

ComputationalSharedFacility

ComputationalChemistry/MDCFDFEA 6,288 750

750-1,000 111

UniversityofOxford Arcus-A

1,728

2,000-5,000 55

UniversityofOxford Arcus-B

5,440 4322,000-5,000 538

UniversityofOxford Arcus-GPU

12

2,000-5,000 146

UniversityofPortsmouth Sciama

fundamentalphysics,cosmologyandastrophysics 3,704 740 100-200

UniversityofStAndrews wardlaw

MHD,Astronomy,Chemistry 3,510 150 200-500 33

UniversityofSussex Apollo

Physics(Astronomy,Cosmology,Particle),Engineering(CFD),Informatics(ComputationalNeuroscience,Adaptivesystems,NaturalLanguage) 3,248 560 100-200

HEITOTALS 127,852 13,318 30,430 2,635

Page 88: Report of 2015 UK National e-Infrastructure Survey of HEIs ... · 16. Regional and national facilities should have an industrial engagement strategy including metrics that permit

NeiSurvey2015ProjectDirectorsGroup(PDG)

AppendixJ–Hardware:Summaryofthesurveydata 88

Totalnumberofprocessorcoresinthesystem

TotalusablestorageforHPC

users(TB)

Numberof

registeredusers

TheoreticalPeak

Performance(Tflop/s)

GRANDTOTALS 1,209,504 286,720 67,680 16,815

(upperbound)