Coalfire Comments on Penetration Tests for Iowa Judicial Branch

(coalfire.com)

100 points | by ajay-d 1678 days ago

12 comments

  • sullivanmatt 1677 days ago
    Here's the scoping doc: https://iowacourts.gov/static/media/cms/Rules_of_Engag_E9D80...

    Some highlights include authorization to attempt entry by tail gating, lock picking, place devices once access has been gained, etc. It's a total vindication for Coalfire (IMO).

    • mike_d 1677 days ago
      Page 3, the scoping section clearly lists the three addresses where physical testing would take place. Page 5 also lists it on the timeline, which based on other notes about not testing specific days - the client would have absolutely reviewed. Page 12 is a client questionnaire where they answer physical security questions.

      People do get arrested doing this type of work. It hasn't happened to me, but it has happened to people I know. Usually it is quickly and quietly resolved without the press being involved.

      What I believe is at issue here is authority. Can a state entity authorize testing of county buildings? Some people just assume "the government" is one entity. This same type of issue arises in corporate work as well - can a tenant authorize testing against a landlords building? You need to get both to agree.

      (Disclaimer: I do this type of work so I might be biased)

      • kerng 1677 days ago
        Pentesters sometimes are leveraged as pawns in political games in organizations - seems similar with government.

        I think you are correct, the question is if the right stakeholders authorized this. You can't break into a 7/11 because some person working there authorized it. Authorization needs to be from all proper stakeholders.

        • danShumway 1677 days ago
          If the county is correct, the Information Security Officer for the Iowa state department payed someone to break into someone else's property. Why isn't there an arrest warrant out for them? If I payed someone to break into a 7/11, I strongly suspect I would get a visit from the police.

          Yes, the two employees could be liable for accepting a request from someone who obviously didn't have the authority to give it. But they wouldn't be the only people liable, and it seems silly to claim that they would be the first people liable.

          Why is it that the only two people arrested are those who had the least amount of responsibility and ability to check on the authority of the State department?

          • dsfyu404ed 1677 days ago
            >Why is it that the only two people arrested are those who had the least amount of responsibility and ability to check on the authority of the State department?

            That outcome is why the word "pawn" is being thrown around.

          • kerng 1677 days ago
            I tend to agree with your obervation.
        • alteria 1677 days ago
          If this is true, is it Coalfire's fault for not verifying and are the two testers SOL with the burglary charge?

          Then is it also the fault of the two testers in this case that they too did not verify that all the correct stakeholders were brought in?

          It would suck to be in the position of the testers having little or no recourse.

          • larssorenson 1677 days ago
            The previous discussion on HN highlighted Mens rea with regards to establishing intent, i.e. they didn't have criminal intent (they went in with the understanding that they had full rights and privileges to be there) and would probably (hopefully) be vindicated in a court of law. There is still a matter of CoalFire vs State / Local govt, where CoalFire might be liable for not confirming that they had the proper authorization to dispatch their testers (which is part of the state vs local issue that's ongoing).

            IANAL but I've been tracking this pretty closely since I'm also a pentester; everything seems to reasonably indicate that the two guys should be released, but someone else is likely ending up in a court over this.

          • iforgotpassword 1677 days ago
            I'm very hesitant to blame the two individuals. While you should have basic understanding of the legal circumstances when working this job, you should be able to trust your higher ups that the paperwork is legit and thorough. A company the size of coalfire certainly has dedicated personnel just for dealing with that.
            • alteria 1677 days ago
              I am too, but was curious what degree of responsibility is expected of them by the law. i.e., is deferring to whatever sales/legal drafted enough of a defense, or something else.

              I work in professional services (although not pentesting), so I curious about what would happen to me if I was in a similar position.

            • kerng 1677 days ago
              Agreed - very little blame goes to the pentesters, they didn't act with malicious intent as far as described. It's the higher ups on both sides that are accountable for the dilemma.
        • 0xDEFC0DE 1677 days ago
          >Pentesters sometimes are leveraged as pawns in political games in organizations - seems similar with government.

          How?

          • gotodengo 1677 days ago
            I've had clients tell me to hit certain known vulnerabilities specifically.

            I can see a situation where the decision to stay or swap off a system is being debated. If one party can send in testers and call out some vulns that might play to their hand.

            • tptacek 1677 days ago
              There's kind of a difference between clients and stakeholders gaming a pentest or spinning its results and pentesters not being authorized to test their targets. Even with basic web pentesting, rules of engagement particularly around which targets you're authorized to test is a big deal. This is a hell of a SNAFU.
              • gotodengo 1677 days ago
                Absolutely agreed. Especially since, in spite of my disagreement with the handling of things, the sheriff may have a point in the authorization angle.

                My initial comment pointed more generally to an example of politics within a company though.

                • tptacek 1677 days ago
                  Like: a scenario that comes up all the time in ordinary web application testing: your authorized target interacts with a third-party API, for which you are not authorized to test. Pentesters generally get this right, because if you get it wrong, no matter what your client tells you, you're liable. (Indemnification may come into play here, but it won't matter criminally).
      • abtinf 1677 days ago
        If the scope of work document is invalid, then the current charges are the least of the involved party’s concerns: the document is proof of conspiracy to commit federal felonies. Many of the pentesting actions would violate hacking laws. The conspirators would include every state official and employee who reviewed and approved the plans, any county employees involved, and anyone else who helped put together the deal.

        If the current cases were to actually go to trial, then even the most incompetent attorney would start asking for subpoenas of everyone involved, likely reaching into the highest political and law enforcement offices. These are people who can exert tremendous pressure in order to evade having to give sworn testimony.

        I don’t know if the local prosecutors understand the massive political retaliation their are inviting.

        • tantalor 1677 days ago
          Trolling?

          > Many of the pentesting actions would violate hacking laws.

          No. Just like the physical penetration, if you are authorized by a legal authority to access a computer system then you aren't breaking any law when you penetrate it. The question is whether these parties were in fact a legal authority.

          • abtinf 1677 days ago
            What a bizarre interpretation of my comment.
    • nekoashide 1677 days ago
      They specifically stated that this access would only be between 6AM and 6PM MST or "Normal Business hours". Not seeing anything that's allowing them outside of these hours.
      • scrumper 1677 days ago
        "Expected to be" is not the same as "will only be", and that's the sort of thing that gets decided by a judge. (The existence of the additional charges language does weaken that argument a bit since if "expected to be" really was meaning that things could happen outside of those hours, then you wouldn't need to reference those MSA terms.)

        IANAL, but have written more than my share of SOWs and contracts of this type. Drafters tend to always default to the position of greatest optionality for them. Hence, "expected to be".

      • foota 1677 days ago
        I felt like that was more about billing than limiting the scope, but you're right that there's seemingly a conflict.
      • alteria 1677 days ago
        Which is interesting since they said that the physical pentesting could be "during day and evening," but you're right. Haven't seen anything affirmatively allowing them to conduct outside of 6am-6pm MST.

        They DID have permission to lockpick, and maybe the state asked for testing after 6pm MST but hasn't released the request (per the blurb below)

        "Requests for testing outside this time period outside the above may result in additional charges per the terms of the MSA"

    • jcims 1677 days ago
      I don't know about total vindication. They may have had all the right things on paper (TBD), but someone could easily have easily been killed and I imagine their SOP will change after this.
  • equalunique 1677 days ago
    There is a legend from the time of my Uncle's tenure at the US DOJ. During the Clinton administration, he hired so-called hackers he met at DEF CON to conduct a pen test of an immigration processing center somewhere around New England. The hackers were given some form of "get out of jail free card" for use during the pen test. In spite of it, they were arrested anyway by the overzealous administrator of the center. My uncle's group in the DOJ had a hard time getting those hackers out of jail, and when they finally came out, they were quite mad, since the whole fiasco had put their permanent records at risk of a bad mark. The pen test project was still on, and it seems they went to extra lengths to exact their revenge on that overzealous administrator. As proof of their total compromise of the immigration processing center, the then Attorney General Janet Reno received in the mail from a green card for a Kang G. Roo. Subsequently, said administrator was demoted and reassigned to some cold desolate part of Alaska. (So the story goes, anyway.)

    Edit: The "reassignment" may have led to an almost-immediate resignation.

    • metalliqaz 1677 days ago
      sounds believable until the Alaska part
      • equalunique 1677 days ago
        I have no idea what actually happened to the former administrator, but I'm told that this hyperbole is method for forcing Special Executive Service employees to resign. There are many hoops to jump through if the intent is to fire them, but part of being an SES includes not having much leeway when it comes to where you are assigned. So by forcing the person to work in some unwanted location, they tend to almost always submit their resignation instead. Where they were reassigned to I don't know for certain, but that's the tactic I'm told was used.

        I suppose "forced to resign" might have been a more belivable way to end the story, but supposedly the mechanism by which that happens involves reassignment to some unwanted location.

        • mirimir 1677 days ago
          The first EPA assessment of dioxins started under the Carter administration, but wasn't completed until the first or second year of the Reagan administration. The lead author ended up reassigned to a tiny branch office, and working in basically a broom closet. Or at least, that's what he said, when I tracked him down to ask questions.
      • GhettoMaestro 1677 days ago
        Being "sent to Alaska" is really just a saying that the organization is super-super-super pissed at you, and desires to send you to the crappiest location (job) possible in an effort to make you leave the organization.

        EG: He fucked up so bad, he will be manning a remote radar station in Alaska for the rest of his military career.

      • dsfyu404ed 1677 days ago
        "Demoted and sent to Alaska" is shorthand for "some non-specific shitty assignment" in this context. Nobody thinks he was literally sent to Alaska.
  • Slenth 1677 days ago
    This article [1] seems to imply the reason for the arrest is a disagreement between the county sheriff's department and the state as to who has the authority to sign off on them attempting to break in to the building.

    [1] https://www.desmoinesregister.com/story/news/crime-and-court...

    • danShumway 1677 days ago
      Importantly, in this story it's confirmed that the pen testers were carrying documentation and phone numbers of people in the State department who could (and did) confirm their stories.

      When this story first broke, there was speculation that, "this is why you carry a get out of jail free card." But if this story is true, the testers did everything right, and the deputy just decided to jail them anyway.

      • mike_d 1677 days ago
        I know physical testers who will carry a fake "get out of jail free card" that lists their own people as the point-of-contact so they can highlight the lack of verification as a weakness. If it does get verified they have a "backup" real one.

        It is possible they attempted this and the police were none too happy being lied to.

        • danShumway 1677 days ago
          It's not possible in this case, unless the news source above is misreporting the story.

          > "I advised them that this building belonged to the taxpayers of Dallas County and the State had no authority to authorize a break-in of this building," Leonard wrote in the email.

          > Leonard wrote that he then called the state employee to tell him his contractors had been arrested and that he didn't have the authority to authorize this.

          If Leonard had called their point-of-contact and it had been fake, it would be weird for him not to lead with that detail.

      • malka 1677 days ago
        Unsurprisingly, cops are assholes
        • mxuribe 1677 days ago
          It isn't only that - there are both good and a$$hole cops...But, it seems in the U.S. there isn't a consistent governance of law enforcement across all levels of government, nor across the whole geography of the nation...leading to some (many?) cops abusing their power.
      • kerng 1677 days ago
        It's not good enough to have an authorization- you need to have authorization from the proper stakeholder!

        That is the dispute here.

      • walshemj 1677 days ago
        The downside of having such balkanised (in that some many individual entities have their own police force) police service I suspect.
      • metalliqaz 1677 days ago
        There is going to be a lawsuit and Iowa is going to settle.
    • jumelles 1677 days ago
      > One man told a deputy that they were conducting a vulnerability study and handed the deputy a piece of paper, which he described as his "get out of jail free card," containing the names and contact information for three state employees, [Dallas County Sheriff Chad Leonard] wrote.

      > A police sergeant called one of the state employees, who confirmed what the men said: that this was a legitimate contract and that the men should be let go, according to the email.

      > "I advised them that this building belonged to the taxpayers of Dallas County and the State had no authority to authorize a break-in of this building," Leonard wrote in the email.

      > Leonard wrote that he then called the state employee to tell him his contractors had been arrested and that he didn't have the authority to authorize this.

      > The state employee disagreed and asked Leonard not to tell other sheriffs, wrote Leonard, who said he responded by saying he was going to tell every sheriff.

      It's not just an implication, the police just decided to arrest them anyway.

    • jtbayly 1677 days ago
      People keep saying it's the sheriff's fault for not listening, or being on a power trip or something and referencing this article. But this quote from the article is pretty important and implies that Coalfire exceeded what was allowed in the contract.

      "According to the agreement, all Coalfire tests were to be conducted between 7 a.m. and 7 p.m. Monday through Friday. But tests could be carried out outside that range if requested through a change order.

      The break-ins in Adel and Des Moines were investigated around midnight on separate days.

      No change order was released that would have indicated the judicial branch wished the tests be carried out at night."

      • rootusrootus 1677 days ago
        Except the sheriff didn't make the argument that it was outside the time window, he said the state did not have the right to authorize a break-in attempt on a county courthouse.

        My guess is the state will shortly remind the county about government hierarchy.

        • nradov 1677 days ago
          State government agencies can only override county governments when there is specific state law allowing them to do so. State bureaucrats can't legally just order a sheriff to do whatever they want.
          • pjlegato 1677 days ago
            It's the other way around: US counties have any legal authority only at the pleasure of their state government, as expressed in its constitution or by statute. They are only immune from state government overrides of their authority to the extent there is a specific law prohibiting the state from overriding their authority.
            • nradov 1677 days ago
              That is incorrect. States do have the power of preemption, but state officials can only issue directives to local government officials when there is a specific enabling state law. They don't have dictatorial powers.
          • bsder 1677 days ago
            > State government agencies can only override county governments when there is specific state law allowing them to do so.

            I think that's only true for a few old states with some oddities dating back to grants from The Crown.

            For everybody else, I'm pretty sure the local government is mostly at the whim of the state government.

            Now most states don't do that because it screws things up badly. However, there have been some high profile cases in Texas, for example, where a state judge comes with the Rangers, displaces the local judge and starts arresting everybody.

            • nradov 1677 days ago
              That is an entirely different and irrelevant example. The Rangers are specifically authorised to arrest people throughout Texas under state law. However they generally cannot issue a legally valid order preventing a county sheriff from arresting a suspect.
          • rootusrootus 1677 days ago
            Perhaps not, though I am unfamiliar with Ohio's state constitution.
        • jtbayly 1677 days ago
          Sure, but it’s not a cut and dry full exoneration of Coalfire, like everybody is claiming. They clearly exceeded the bounds of their contract if this is true.
      • 0xDEFC0DE 1677 days ago
        The scoping doc link in this thread pretty clearly indicates the address where this happened.

        The fault seems to be with the person who authorized the physical portion in the contract not actually having the authority. And unfortunately that means Coalfire employees are caught in the crossfire.

  • matthewdgreen 1677 days ago
    Why is this still going on? It made sense before when there was a possibility of confusion, but at this point it is at worst a mistake, not someone with intent to commit a crime. Are charges still being pressed?
    • doctoboggan 1677 days ago
      The question here is not whether they were contracted to do the pen test, I think everyone agrees they were. The question is whether those who contracted them had the authority to authorize it.

      If someone from craigslist give me a contract to break into a house to test their security system, but it turns out the owner of the house did not know about the contract, who is at fault for the break-in?

      I do think that Coalfire acted in good faith here, but it is a complicated situation.

      • mcherm 1677 days ago
        > The question here is not whether they were contracted to do the pen test, I think everyone agrees they were. The question is whether those who contracted them had the authority to authorize it.

        No, the question is whether the prosecutor believes beyond a shadow of a doubt that the accused had the requisite intent to be committing burglary. If not, the they have an ethical obligation to drop the prosecution.

        • doctoboggan 1677 days ago
          You are correct, I didn’t know burglary was a specific intent law.
    • nathanyz 1677 days ago
      Agreed, I mean these guys are now pawns in some political game. But these are people's lives. Let them off the hook and then you can commence with your dick measuring contests.
    • benmmurphy 1677 days ago
      I think the better question is why hasn't someone in State been arrested. If I contract a lock smith to break in a door I don't own and provide a bunch of misleading information about it I'm pretty sure I would be the one facing charges in the end and not the lock smith.
    • metalliqaz 1677 days ago
      Power is a helluva drug.
  • exabrial 1677 days ago
    Sounds like yet another over-zealous prosecutor hell bent on putting non-violent [and non-criminals] behind bars.
    • sjcoles 1677 days ago
      Nah just an idiot sheriff having a power trip[1].

      [1] https://www.desmoinesregister.com/story/news/crime-and-court...

    • jerf 1677 days ago
      That seems unlikely. How would prosecuting this case rebound to a prosecutor's advantage in anyone's eyes? "Ha ha, we contracted with these people to do security work for us, but we sure showed them when we hit them with criminal charges for the things we're paying them to do! Nothing says I'll pursue justice like paying to entrap people and prosecuting people with the opposite of mens rea. Elect me or promote me as appropriate, I'm openly a treacherous lunatic!"

      Incarceration is a tool for over-zealous prosecutors, not a terminal goal. That tool won't help them advance their actual goals in this case.

      (Of course, tools that get used a lot can accidentally become terminal goals if a non-self-reflective agent gets too cognitively lazy. Still, the intelligence bar for "prosecutor" is probably high enough we should credit them with generally knowing what they're doing. And even if they were being this lazy, their full attention would be brought to this case when it first hit the media.)

      • PeterisP 1677 days ago
        The goal of the sheriff and the prosecutor is to stand tall in their ongoing conflict with the state institution, of which this pentest is just the latest event.

        It's empathically not "we contracted with these people to do security work for us", it's that "our enemies from the Iowa state contracted these people to try and screw us".

        They succeeded in getting some of these enemies caught, so they will make a point to their local voters as "we sure showed those bastards from the state agencies that they have no right to come here and shove their noses in our business" (which they've already explicitly stated to local press in pretty much the same words), that's a win already for a certain class of the local voters.

        Their goals explictly include making it harder for 'enemy agents' to operate. If they use every technicality within the bounds of the law to make it as prolonged and difficult as possible, they're making a statement that attempting to attack them in this manner will result in retailiation - if it makes it hard for Iowa state to employ other pentesters in the future, excellent, mission accomplished, that fits the goals of these local agencies very well. They treat this mission as an explicit attack against them, and they want to deter attacks like these with whatever power they've got - which includes the ability to arrest people.

    • rootusrootus 1677 days ago
      Or just an relatively innocent combination of dumbfuckery and confusion. The prosecutor probably already wants to drop the whole matter and try to forget about it, but it's political now.
    • jnbiche 1677 days ago
      Yep. Someone has his or her eye on a higher office.
      • jstanley 1677 days ago
        Can you ELI5 why putting pentesters in prison would help a prosecutor get promoted to a higher office?

        Wouldn't it just piss off (among others) the very people who have the authority to promote the prosecutor to a higher office?

        • fotbr 1677 days ago
          Prosecutor gets to go trumpet a victory over those who would show such disresepect to the law as to try to break into the courthouse itself (or various press releases to similar effect).

          The "very people who have the authority to promote the prosecutor" are the voters, who, for the most part, do not understand what pen-testing is, or that it exists, or that it is (or can be, when done correctly) a legitimate thing.

          ELI5 A person is smart, people are stupid. Voters are people, and details get lost in campaigns.

        • ptyyy 1677 days ago
          Perceived "easy win" for conviction rate due to a fundamental lack of understanding of the issue. This backfired hilariously.
      • itomato 1677 days ago
        Or an interest in disruption at any and all available opportunities.
  • Animats 1677 days ago
    One of the many books on CIA training describes how they handle this. The CIA has written agreements with law enforcement in the areas where they do training exercises. Trainees are given a number to call. If they call it, someone from CIA HQ comes over, with, as one trainee put it, the "rumored but never seen get out of jail free letter".

    This usually means the trainee failed the exercise.

  • sandworm101 1677 days ago
    >> It's a total vindication for Coalfire (IMO)

    But think about this from the perspective of the cops. The contract can get coalfire out of any liability for damage done to the building and any potential break and enter. That is consent between contracting parties. But an alarm was set off. The police were called. This isn't exactly a case of them filing a false police report, but the police were indeed called under false pretenses.

    I used to work in a building with remote monitoring and extensive security, including armed response (military). We did these tests monthly. But as soon as the alarm was triggered, someone was on the phone to the military police. If their supervisor decided to roll the cars and test his officer's response time that was with his permission. We would never, ever, have insist that cops stop what they were doing, possibly something dangerous/real/important, and physically respond to our not-real security test.

    • tantalor 1677 days ago
      Who cares about the cops? They don't have any role in prosecution.

      "In the criminal justice system, the people are represented by two separate, yet equally important, groups: the police, who investigate crime; and the district attorneys, who prosecute the offenders. These are their stories."

  • heyflyguy 1677 days ago
    Are these bot comments? Looks like a simple contractual misunderstanding, probably exasperated by a bureaucratic communication issue of some sort. I'm sure we'll discover that coalfire had ducks in a row and Iowa didn't know what they bought. Nobody communicated it and here we are.
    • cududa 1677 days ago
      It so far appears the county sheriff is behind this, insisting that the courthouse belongs to the county, and the state had to right to authorize this test.
  • aaron695 1677 days ago
    Any action that causes the police to investigate and leaves them continuing to investigate what they think is a crime, is irresponsible. (Polk break in)

    Using locking picking tools is destructive (it technically causes damage)

    "Social Engineering" (What the physical access section is under) to gain physical access is not lock picking, although there's no evidence they used the tools? and not opening doors or windows they are not allowed to open, when no one is there.

    This is stupid, yes, the men should not be put in prison, but equally Coalfire messed up bad, maybe they verbally got the ok then it's back onto state who messed up bad.

    What if the cops had drawn weapons. Someone does need to get into some sort of trouble for this.

  • stevespang 1677 days ago
    I predict there will be a lawsuit for damages by the two testers arrested.
  • zelon88 1677 days ago
    Here's a post about the subject that's actually worth reading. https://www.scmagazine.com/home/security-news/vulnerabilitie...