Print 🖨

The House Committee on Urban Affairs met to on April 5 hear invited testimony over interim charge #2:”Identify and address potential gaps in cities’ cyber security policy and ensure that personal information held by cities and other municipal entities is secure.”
 
Panel 1
JJ Rocha, Texas Municipal League

  • TML hosted a cyber security workshop with Chair Alvarado a few weeks ago and conducted a survey of 144 cities within the TML, survey attempted to target cities with a high risk of cyber security issues
  • 56% of city government surveyed is “ready” for a cyber security threat, as judged by factors including backing up data, use of anti-malware software, etc.
  • Less than 40% of respondents have a cyber security plan, likewise not many cities had cyber security issues built into their emergency plans
  • TML’s survey found that even though many cities did not have plans, information about cyber security threats was very available, also concluded that this is an education problem rather than a legislative problem
  • TML is willing to host an education service on their website, conduct workshops and training as needed, and conduct a cyber security seminar at the annual TML meeting
  • Alvarado – Thanks TML, knows TML has made a lot of progress on cyber security education, highlights that San Antonio is on the “forefront” of the cyber security issue and one of the city councilman has called for a cyber security plan to be drafted
  • Schaefer – Where any of the cities surveyed rural?
    • Yes, some, but most were urban
  • Schaefer – Has TML found any statutory reporting requirements? What legal responsibilities does a municipality have upon data breach?
    • Unaware, but will work with Alvarado to get this information
  • Alvarado – Looking at the San Antonio plan it calls for 3 cyber security audits and plans for municipalities, would like to get a handle on this before next session

 
Panel 2
Dr. Art Conklin, University of Houston

  • Here to address the broad issues facing cyber security and power
  • Power generally has many consumers and few suppliers, to control pricing with a glut of consumers, governments have used regulating commissions
  • Main question facing cyber security issues is who responds to the problems? It cannot be the PUC as it only controls the pricing of utilities
  • Therefore, utilities, who have all of the crucial equipment supplying power, must take the task of preparing for cyber security problems
  • Power grids have been attacked in the past (e.g. Ukrainian grid attack in Dec. 2015), Texas’ power grid is not prepared for attacks of a similar nature
  • The last major revision of grid regulations happened in 2003, before cyber security was considered a serious issue
  • Individual grid preparedness varies drastically based upon geography and controlling authority
  • However, Texas’ advantage is that power in Texas is controlled by large statewide entities with few exception
  • Laws will not fix the issue, attackers already disregard laws
  • Rather, legislature should explore ways to let utilities respond to threats in the quickest way possible and ways to alert utilities that they are vulnerable to attack
  • Alvarado – Appreciates testimony over Ukraine attack, important to consider the possibility even though Texas could be “more advanced”
    • Information available about Ukraine is largely wrong
    • While Texas’ grid is more advanced, Ukraine was able to regain control of their grid by manually fixing the problem at different power nodes, Texas does not have the manpower or skillsets for this and is thus vulnerable to similar types of attacks
  • Schaeffer – Do non-nation attackers have the capability to attack our power grid?
    • Soon, what happened in Ukraine will be doable by IT grad students
  • Schaeffer – Can non-nation attackers take down small electrical providers in the US?
    • Large companies are a lot less vulnerable, but can still be attacked
    • However, attackers do not go for hard targets, typically look for vulnerabilities in the weakest entities
  • Schaeffer – Do you look at other threats to the grid?
    • Yes, however, grid is not the biggest issue, water is
  • Schaeffer – So what are the threats to public water systems?
    • Threats to power, water, gas, etc., are fairly similar across the board
    • Electricity is appealing, but it is difficult, water is a lot easier to manipulate (i.e. much easier to turn off a pump than it is to discern the workings of a power grid)
    • Another issue with water is that it is a collection of small utilities without the central oversight power has
  • Alvarado – Can you expand on how distribution matters the most?
    • Distribution is where the state gets its power, above ground power transfer systems are vulnerable
    • Utilities are able to reroute power around disabled stations
    • Distribution issues are most common and easy to fix, though while cyber threats are rare, they are much more difficult to fix
    • Distribution problems happening in sequence can be a lot more difficult to deal with, however (“death by a thousand cuts”)
  • Schaeffer – What about a coronal mass ejection (solar events)? What is the probability of EMP events
    • Does not worry about naturally occurring EMP events
    • Likewise does not worry about coronal mass ejections, would likely kill everyone concerned, not a realistic worry
    • Cyber attacks will likely happen outside of US borders
  • Schaeffer – So EMP is very low probability, high impact, don’t worry about it?
    • Not saying don’t worrying about it, rather focus on things that the state is able to handle

 
Brian Lloyd, PUC

  • Jurisdiction of PUC is limited when it comes to municipalities or cooperatives, elected boards of each tend to regulate
  • PUC does have limited regulatory power customers of systems if they live outside the city limits
  • Inside ERCOT, PUC regulates wholesale transmission rate and this does include municipalities and cooperatives
  • If a municipality or cooperative generates power within ERCOT, PUC also have a role in ensuring compliance with ERCOT standards
  • Electric systems are very complex, however, this complexity can lend strength as systems are generally highly redundant and can prevent or restore issues
  • PUC rules require transmission to be continuous and adequate, also requires maintenance of emergency response plans and ensuring security of customer information
  • Public utilities tend to adopt similar requirements for their grids
  • PUC contracts with federal partners who provide training and security standards
  • Impractical and ill-advised for regulators to tackle security problems in place of utilities, more appropriate to provide as much information, education, and training to utilities as possible
  • Electric power tends to conduct drills and exercises more frequently than other utilities
  • Alvarado – What are your responsibilities regarding cyber security with utilities?
    • Falls under general regulatory authority, PUC regulates that utilities must have continuous and adequate service and must have cyber security emergency plans
  • Alvarado – Is the PUC looking at adopting a plan or regulatory scheme for the utility sector?
    • PUC typically works closely with utilities on this, utilities are required to have a response to cyber security threats
    • Does not think any utility has failed to prepare for threats, though unsure what regulations could be made given the shifting nature of the problem
  • Alvarado – Do you think that the current cyber security regulations are sufficient?
    • Utilities can speak to the sufficiency of their practices
    • Difficult to think about how government could step in and micromanage this issue
  • Alvarado – Certainly doesn’t want to micromanage anything, but someone needs to be accountable
    • Current law states that utilities are required to have plans for this
  • Alvarado – Can’t continue to think of this in “a bureaucratic way,” trying to prevent a patchwork approach to cyber security with too many different agencies handling too many different areas
    • PUC is very sensitive about adding layers of bureaucracy
  • Alvarado – Would want to add layers intelligently
  • Hunter – Is regulation of cyber security in utilities an issue?
    • Conklin – Yes
    • Lloyd – Yes, it is an issue for utilities
  • Hunter – Also believes this is a serious issue, what do you propose the legislature do? Last thing you want to do is “prepare after the hurricane comes”
    • Conklin – Cannot micromanage this issue, not sure anyone knows the answer to this and perhaps a study of appropriate parties followed by a strategy to allow those parties to handle the cyber security issue
  • Hunter – So your thought is a legislature coordinated effort determining what can be helped and who can coordinate to tackle this
    • Difficult to think of what law could be made to handle cyber security better than what the big utilities are already doing
    • For water utilities, might be good to have a discussion over which agency should have authority over cyber security (e.g. TCEQ controlling pumps and PUC controlling rate setting issues)
  • Hunter – Promote ocean desalination in Texas, but now understands that cyber security will have to be considered for this too
  • Hunter – So you agree that state government should coordinate with the private sector to handle this issue?
    • Conklin – Unsure if he and PUC are in agreement, but the financial sector handles cyber security better than the government
  • Hunter – Basically looking for a framework to handle problems
    • Yes, financial services deal in many issues and understand how to work as a cooperative, models like this would be very helpful
    • All major critical systems use the federal iSAC model, however functionally the financial services industry leads the way in cyber security
  • Alvarado – How do balance security requirements, regulations, and rate requests
    • Not a bright line answer, PUC expects utilities to do what it takes in all arenas to ensure continuous and adequate service so long as it is cost-effective and reasonable (e.g. cannot move entire infrastructure underground as this is prohibitively expensive)
  • White – You have standards that you expect providers to meet, but is there oversight making sure they are complying?
    • PUC really has authority over roughly one dozen utilities, PUC does a lot of informal collaboration and conversations to make sure these utilities are taking cyber security threats seriously
  • White – Does not want over regulation, but wants something laying out general standards for utilities to operate under
  • Schaeffer – Given a municipality that runs a water system, who tells this municipality how many back-ups it must have?
    • Municipalities are largely autonomous, certain things like water quality or outages may fall under TCEQ
  • Schaeffer – So if we are trying to figure out risks and figuring out basic issues, can we not simply look at standard requirement for municipalities that dictates what the emergency supplies are?
  • Alvarado – Does not think that cities have something in place
  • Schaeffer – Some private industry power backups only last days, finding out what standards and rules govern emergency preparedness for water systems could be a huge project itself
  • Bernal – Great argument for solar power
  • Bernal – Legislature is not made up of cyber security experts, but people ask the legislature what ensures that public systems are protected or can recover from cyber attacks, seems that committee is hearing that they should take this preparedness on trust
  • Alvarado – Thinks general public assumes utilities are well-prepared, regulations are in place, and government is supporting these preparedness measures
    • Conklin – For hurricanes, the emergency process is continually being improved, cyber security will likely work the same way – Texas will not know what is needed until the system is tested
  • Bernal – If we have to gather the greatest minds together to figure this out, where do those minds come from?
    • Conklin – Starts with legislature, they can lead and enable the public
    • However, will need to have utilities, agencies, etc. involved, this problem involves more than just the municipalities
  • Bernal – So the next step is to study and choose the right people to lead the charge on the issue
  • Anderson – Some financial services track attempts to attack their systems, does the state or do agencies track attacks made?
    • PUC does not track it at their level, all utilities under their authority have extensive cyber security, protections and methods are being updated constantly, ERCOT has measures, etc.
    • PUC struggles with this issue, what it should mange and track
  • Anderson – Number of attacks would be useful for regulators to know
  • Anderson – Who is responsible for fixing cyber security problems?
    • Utilities
  • Anderson – If they do not have the expertise or resources to fix these, who helps them fix these issues?
    • Conklin – For physical impairments, utilities have cooperative agreements, could mirror this for cyber security
  • Anderson – Have to assume that an attack will be effective at some point, would be good to know who should come in and support recovery
    • Technically DHS
    • Conklin – If you’re waiting for DHS to help, this would be a situation like Katrina – DHS cannot possibly respond to all problems Texas would have
  • Alvarado – Seems to be a consensus that a group of stakeholders needs to be formed

 
Panel 3 – Energy Utilities
Michael Goin, Austin Energy

  • Austin Energy (“AE”) now has an automated work management systems and other electronic communications means to maintain and manage their grid,
  • PIRs can constitute a security risks, PIRs are onerous and require expenditure of money and personnel time, private energy providers do not have to deal with this
  • Austin Energy has been subject to PIRs that have released potentially sensitive information
  • Hiring is also a difficulty in the energy marketplace, Austin Energy has a difficult time offering competitive salaries
  • Austin Energy follows a Department of Energy risk management policy that fits closely with the corporate risk management policy, operating bodies at each management levels ensure swift and effective communication
  • NERC plays a huge role in regulating the industry, Austin Energy is NERC compliant, while compliance is not considered security, compliance is baked into security protocols
  • Also adheres to Texas code for personally identifiable information
  • NERC standards are very good at incentivizing better security standards and practices
  • State should look at public information act, PIRs can result in security issues and unfair competitive advantages
  • PIRs for industries supporting critical infrastructure should include more specific information and justification and time for sufficient vetting
  • NERC compliance should be easy to maintain when trying to maintain good security
  • Information sharing for security issues is very important between utilities and government

 
Joel Austin, Oncor

  • Oncor has built standards following NIST standards including risk and compliance functions, security operations function, integrated operations, and NERC cyber oversight
  • Oncor follows military standards and practices, military has been spending money on advanced cyber security solutions
  • Oncor believes tracking interior data for threats and attacks is just as important as looking outside an organization, cites the Ukraine attack
  • Oncor attempts to combat the sheer weight of threats by correlating data of passive attacks with trying to determine what human actors would be interested in
  • Most attacks begin with email links or credential leaks that allow attackers in, Oncor tries to train its workforce to combat this
  • Oncor runs a security certification program for grid technology providers, tries to determine what is secure for an electricity supply network, Oncor can ensure its supply is safe
  • Third parties are useful for contracting to test and bolster security
  • Agrees with everything Goin said about compliance with NERC
  • Major challenge to collaboration is how information should be shared and how any entity can determine threats
  • Oncor has been working with national laboratories to develop CRISP, which is a tool that monitors networks and allows for information sharing of federal threat information

 
Ann Delenela, ERCOT

  • ERCOT is not a market participant, but is charged with maintaining reliability, ensuring open access to transmission, ensure retail switching, and managing the wholesale market
  • Sophistication of cyber attacks continues to increase, challenges exist with this and understanding threat actors and their motives
  • Social engineering and manipulation of personnel is also a threat vector, oftentimes more dangerous than direct cyber attacks
  • ERCOT has a dedicated cyber and physical security organization, board of directors has oversight
  • Security standards are often advanced because of regulation, because of crisis, or because of leadership, relying only on regulation is insufficient, relying on crisis is irresponsible, leadership and forethought are the best options
  • In addition to NERC and NIST, ISO27000 exists, all options are great frameworks for establishing a security program
  • Regulatory response can be effective, but takes time and is oftentimes not responsive to the most recent threat development
  • Should demand that systems created by providers have security measures baked in them rather than bolted on after the fact
  • Industry has made an effort to develop information sharing (e.g. E-ISAC)

 
William Whitney III, Garland Power and Light

  • Ever-changing field of technology constantly present new security risk
  • Speedy recognition of threats is essential, would be useful to have a centralized information sharing system that compiles and distributes information (e.g. E-ISAC)
  • Support from organization management and employee cooperation is important to propagate security standards throughout an organization
  • NERC standards are not stagnant, constantly being updates and GPandL works constantly to meet requirements
  • PSAs or other developed emergency information network could be used to effectively distribute information of security attacks
  • Education programs could be effective at instilling good security programs, could defend against phishing attacks or help keep systems up to date and protected against threats
  • Similarly, vendor responsibility is important to protect against threats, protections should be instituted at the vendor and provider level

 
Mike Phillips, CenterPoint Energy

  • Primary challenge CenterPoint faces is with volume of threat information and managing this information
  • Sometimes threat information is irrelevant, determining relevance is onerous
  • Security standards must cover every part of an organization, human element is weakest to cyber threats
  • Culture of cyber safety and prevention is important, as useful as things like OSHA regulations, must start at the top and propagate down throughout the organization
  • Two practices that are effective in lieu of legislation:
    • An organization should identify critical assets and have protection measures as well as recovery plans upon breach
    • Federal standards, such as those developed by the DOE, are applicable to all utilities, helps develop security standards at the organization level
  • In courts, NIST has been used to hold organizations with lax security standards accountable, likewise insurance companies have used NIST to guide their policy standards
  • NERC standards are the minimum, corporations and organizations tend to comply with minimum, standards should be instituted to incentivize better security
  • Mutual assistance programs for cyber security are in development, however standards for this have not yet been hashed out
  • Regulatory controls provide the minimum that companies must meet for security, technical controls are typically industry driven and drive organizations to benchmark each other on security, competition can help build better standards
  • Standards differ for transmission and distribution, distribution tends to be more free with information sharing
  • Important to have standards consistent and able to keep pace with security threats
  • Important to promote good security across the breadth of the industry, majority of security controls can be implemented by small organizations

 
Questions for Panel 3

  • Alvarado – As a regulator, what is ERCOTs responsibility in regulating utilities?
    • ERCOT does not have any responsibility to regulate utilities, ERCOT does provide a forum for education and information sharing to promote best practices
  • Alvarado – Would you support a stakeholder taskforce?
    • Absolutely
  • Alvarado – Where do the utility companies go for mutual aid and assistance in the event of an attack?
    • First step would be to reach out to peer companies, also can contact DHS’s fly away team
    • Would have to contact each and every peer, distribution list helps, but there is no coordinated system
    • Legal might not let an organization release information
  • Alvarado – Is there a minimal set of procedures or guidelines?
    • Whitney – ERCOT has a set of protocols
    • Delenela – ERCOT’s protocols do not speak to cyber security, E-ISAC is working to develop a trustworthy relationship with utilities
    • Whitney – The TRE handles regulation of ERCOT standards and protocols, TRE also audits utilities
    • Phillips – Historically, natural disaster plan has been used to frame a cyber incident response plan, cyber liability insurance requires a breach notification plan
  • Alvarado – What can government do?
    • Austin – Discussion with government is critical and Oncor welcomes this, standard rule set is difficult to apply across all entities within the industry and information sharing is a good solution
  • Alvarado – How often do you audit your systems and who does this?
    • Phillips – CenterPoint has a security operations center to monitor everything in real time, contracts with a third company to perform a weekly sweep, and performs annual audits in addition to the NERC audits
    • Austin – Oncor uses third-party specialists to help handle this,
  • Anderson – The weakest link is the number of users on the network, do you use simulations with friendly adversaries?
    • Yes, used commonly to try and identify weaknesses
    • Simulations are conducted for a number of scenarios, new physical and electronic security training
  • White – Are there warning systems for cyber attacks?
    • Depends, sudden attacks can happen without warning
    • ERCOT highlights that this is why information sharing is so important to allow for timely warnings
  • White – So not one plan or fix can address this, barriers also exist to collaboration across organization
    • Yes, Fusion Centers can address this, but can be further enhanced
  • Schaeffer – ISAC is more of a fusion organization and CRISP is a tool?
    • ISAC is an information sharing entity, CRISP is an information gathering tool managed by ISAC
    • CRISP program is an expensive utility that monitors perimeters of organizations nationwide and submits access information for computer analysis, this can be the warning that Rep. White wanted, however CRISP is not very broadly used
    • CRISP is used by ERCOT, Oncor, and CenterPoint in Texas
  • Schaeffer – So roughly 50 companies participate in ERCOT< but a fraction of those participate in CRISP, is this a cost issue?
    • Rather an issue of laboratory’s ability to deploy analysis
  • Schaeffer – So there is not a Fusion Center for cyber security, can Texas create its own?
    • Can look at enhancing the existing Fusion Center which gives Texas the capability to handle all hazards
  • Schaeffer – So ERCOT can be involved with individual utilities’ reliability and transmission issues, but not in cyber security issues?
    • This is where collaboration with E-ISAC becomes very important to prepare entities inside and outside of Texas
  • Schaeffer – Edison Electric Trade Association is another national organization providing standards, how many utilities participate in that?
    • ERCOT does not have this information
  • Schaeffer – So again this is a situation where standards exist, but entities in Texas do not fully participate?
    • Whitney – So many different entities exist, difficult to choose which an entity wants to participate in or pay for, etc.
    • ERCOT is encouraging participation in ISAC
  • Schaeffer –  Does every ERCOT company have an air gap between operational electric management system and administrative and customer billing
    • Delenela – Would need to be answered by individual utilities
  • Schaeffer – So the security advisor for ERCOT cannot answer this?
    • No, ERCOT does not have visibility into utility procedures like this
  • Schaeffer – You don’t ask them?
    • ERCOT does not, it does not have that oversight
  • Schaeffer – Would you agree that being air gapped is a critical security measure
    • Certainly a best practice, would need to look at past compromises where air gapping can be overcome
  • Schaeffer – So who in Texas has the authority to tell every provider that they must be air gapped? Does anyone?
    • Delenela – That’s a good question
    • Probably not, this is where guidance from someone with authority would be beneficial        
  • Schaeffer – Are all of the Edison Electric Trade Association companies air gapped?
    • Austin – the Edison Electric Trade Association does not regulate, standard practice for larger companies, certainly is NERC, TRE would audit against this
  • Schaeffer – Is there any governmental entity asking companies if there administrative systems are physically separated?
    • If they are not in the transmission business, probably not
    • Whitney – Connection to the outside world is another issue, so air gapping can isolate a billing system, but at some point those systems need to be accessed outside of isolated locations
  • Schaeffer – I agree, but at some point you need to limit that access and there need to be protocols, air gapping is still the best practice
    • Delenela – It is a good practice, but certainly other means exist to provide segmented control
  • Schaeffer – So there is no answer to who has the authority to promote best practices
    • Phillips – Likely PUC regulates utilities for this specific instance
  • Schaeffer – Does anyone in Texas require forensic audits?
    • Utilities do this because it is a best practice
  • Schaeffer – So smaller utilities might not?
    • Correct, if they fall under NERC then there are requirements, but oftentimes they do not fall under regulatory organization
  • Alvarado – Appreciates that utilities are willing to work with the legislature

 
Panel 4 – Water Utilities
Bill Fry, Association of Water Board Directors

  • MUD systems are very vulnerable to cyber attacks, billing systems tend to have protections
  • SCADA operational systems have some protections, mostly provided through third-party providers
  • Association attempts to educate members about cyber security issues, utilizes third-party consultants to help develop and conduct some educational programs
  • Association expects cyber security to be a continuing education matter, water utilities have likely not caught up with other industries in security
  • As more systems become connected through technology, expects board members to demand more robust security measures
  • Boards already try and do this individually, association wants to help distribute this as widely as possible
  • TCEQ regulates majority of water utilities’ operations, PUC regulates rates only
  • DHS requires water utilities to annually report on operational security and report any breach of water system security, though focused on physical breaches rather than cyber security
  • SCADA systems are being increasingly opened up to the internet, this has not yet been addressed
  • Current regulations neither inhibit nor incentivize cyber security policy, TCEQ regulations are largely silent, however, MUD clients are now asking the association to have cyber insurance
  • Each individual district largely decides how much they wish to devote to cyber security
  • Committee should engage homeland security and TCEQ
  • Communication between entities involved with water has been limited so far

 
Teri Pennington, Austin Water

  • Software remains a large vulnerability, legacy systems and resource constraints can prevent updates
  • Vendors also are sometimes reluctant to update systems, wanting a solution to be able to force vendors to update in a timely manner
  • Other security breaches have occurred due to third-party contractor fault, important to have accountability for third-party contractors
  • Important to educate procurement officers that understand the risks in cyber security and have security standards baked into contracts
  • Cities have multiple standards that AW must meet for different types of information security, TAC has requirements for personally identifiable information, all standards come from NIST
  • In some cases, gaps exist in regulations, especially with cloud vendors (at risk data could be located outside of the US and regulations may not apply)
  • Important to look at industry such as credit industries, while no government regulations cover cyber security for certain information, standard exists and was developed by industry participants
  • Private entities have started to rate cloud vendor security practices, a consistent standard given to the marketplace could let companies rate water utilities, this can be very flexible and updated daily
  • Austin Water is part of Austin’s incident response plan and emergency operations center, this framework can work for cyber security
  • Austin is also a member of the MS-ISAC, can provide early warnings
  • TXWARN is also a mutual assistance organization for water and wastewater utilities

 
Margarita Hubbard, San Antonio Water System

  • IT department supports the technical aspects of SAWS, SAWS also has certain web application that allow for consumer outreach
  • SAWS’ SCADA system was audited last year, also covered the network, audit team was not able to penetrate the SAWS network
  • Cyber security evolves on a daily basis and areas that need security vary widely, has had experience with attackers trying to divert payments via phone
  • Increased mobility of field workers is a potential area for vulnerabilities, SAWS has refrained from advanced quickly into this area to ensure the safety of the mobile system (e.g. ensuring data is not retained on any lost tablet or mobile device)
  • Organizationally, IT department system administrators ensures the security of varies aspects of the utility such as ensuring security of software and consistent updates as well as access control of company data
  • SAWS has recently hired a director for network security in response to growing number of threats to develop security policy
  • There are no applicable Texas standards for SAWS to apply to water system security, must develop these policies internally
  • SAWS instead follows NIST standards and enforce monitoring standards for water and wastewater facilities
  • Also trying to promote good personnel standards and a good company security culture
  • Has had difficulty demonstrating benefits of secured devices to field personnel, important to promote this
  • Would be good to have a committee or stakeholder group to recommend good security practice, framework would also be good to develop company policy
  • Would be good to have a central source for security threat and infrastructure information, security measures for infrastructure data are needed

 
Panel 4 Questions

  • Schaeffer – So DHS requires you to report physical incidents?
    • Fry – Yes, but cyber incidents do not have to be reported
  • Schaeffer – Is there any cyber threat information sharing initiative?
    • Pennington – City of Austin participates in the Austin Regional Intelligent Center, city management requires reporting
  • Schaeffer – Is there any statewide policy, is there any kind of Fusion Center? Any kind of cyber reporting to DHS
    • Fry – For cyber security issues, no
  • Schaeffer – Are you aware of any serious cyber security issues with water systems in Texas?
    • Unaware
  • Schaeffer – Are you aware of any serious fiscal incidents?
    • No, billing systems have firewalls, Red Flag system requires reporting
  • Schaeffer – So most water systems are moving to SCADA, is this a big vulnerability?
    • Yes, down the road
    • Also have to consider that many companies use third-party providers to run their SCADA systems
  • Schaeffer – What would a cyber attack look like? What can happen if a SCADA system is breached, can chemicals be released into the water or can water be turned on or off?
    • Chemicals are not released into the water by systems typically
    • Water can be turned off, but redundant alarms
  • Schaeffer – Is there any air gapping?
    • Yes, however billing and administrative systems are combined for the AWBD
    • Some SCADA systems are and some are not air gapped, but it is not easy to transfer from customer to administrative to operational
    • Hubbard – SAWS hired a third-party firm to audit the SCADA system, firm was able to breach the SCADA system via default passwords
    • This was a vendor issue, they had not changed passwords on the vendor software
    • Firm was able to turn water off and also spoof notifications that the water flow was fine
  • Schaeffer – How concerned should the committee be about water systems? Does the committee need to take action?
    • Fry – Water systems are concerned because infiltrations can happen, AWBD considers this a serious issue
    • Water systems are public entities, also involve concerns of public and private information
  • Schaeffer – Is this like the energy systems, i.e. are smaller systems more vulnerable?
    • Yes, mostly a matter of resources, to implement more the price of water might have to rise
  • Schaeffer – So no one requires forensic audits if the local governing authority does not?
    • Correct
  • Schaeffer – How involved is TCEQ in SCADA management?
    • They know we have it, but they do not regulate this, do not check for air gapping or security practices, etc.
    • Hubbard – SAWS has an air gapped system, but possibility still remains that someone could affect flow, SAWS also does visual inspections to ensure flow
  • Schaeffer – How many days can water pumps run with no electricity?
    • Fry – Depends on if backup generators exist, fuel was the limitation of AWBD
    • Without electricity, the average water utility will not have water
  • Schaeffer – So how many days can the average 10,000 person city keep the water running?
    • Depends on storage
    • Ability to run water system on generator was limited, TRW said that resources were not there to run generators even with a statewide bill proposal
  • Schaeffer – Understands that there is a cost involved, but generators are not hugely expensive, is it your testimony that away from the coast, most water systems do not have backup power generation?
    • Yes
    • Most water systems have some kind of means of mechanical pumping
  • Alvarado – And how many days can that survive?
    • Again depends on the storage capacity
  • Schaeffer – And without ability to pump from storage, there is still a water issue, maybe have a county-by-county solution where people may travel a reasonable distance to obtain water
  • Alvarado – Hopes everyone would want to participate in a stakeholder group, would like to get this work done soon to be prepared for session
  • Schaeffer – Would like to bring in emergency management professionals to develop an emergency response while security standards are worked on