- Video report by ITV News Security Editor Rohit Kachroo
Terror plots are escalating to the point of violence in a matter of days after a "dramatic" jump in the scale and pace of the threat facing the UK, the head of MI5 has warned.
In rare public comments, Andrew Parker said that the danger the country faces from terrorism is at the highest point he has seen during his 34-year intelligence career.
In separate comments made to ITV News, Mr Parker insisted that despite five terror attacks being carried out since March resulting in the deaths of 36 people, 2017 has not been a year of failure for the security services.
"We've seen a dramatic up-shift in threat this year," Mr Parker said in the first speech he has made since Britain was hit by a wave of terror attacks in 2017.
He continued: "Today there is more terrorist activity, it's coming at us more quickly and it can be harder to detect."
Mr Parker said that security services and police are operating at a scale "greater than ever before", with well over 500 live operations involving around 3,000 individuals, but conceded that there will be some violent extremists "not yet known to us at all".
Agencies must also assess the risk posed by those returning from Syria and Iraq, and the "growing pool" of more than 20,000 people looked at as part of previous terror investigations.
Mr Parker said the increase in the scale of the terror threat facing the UK is being driven by the "murderous strategy" of so-called Islamic State and online propaganda, and warned that the threat is "more diverse than I've ever known".
These plots ranged from those "developed here in the UK" to those "directed from overseas", and from those involving "lengthy planning" to "spontaneous attacks".
He continued that "extremists of all ages, genders and backgrounds are united only by the toxic ideology of violent hatred that drives them.
"These threats are sometimes now coming at us more quickly - whether crude but lethal attack methods, for example using a knife or a vehicle, or more sophisticated plots, when in today's world terrorists can learn all that they need online to make explosives and build a bomb."
He added that attacks can sometimes accelerate "from inception through planning to action in just a handful of days", adding: "This pace, together with the way extremists can exploit safe spaces online, can make threats harder to detect and give us a smaller window to intervene."
Among the plots seen in recent years some have had a "very short life from first inception to intent to act", Mr Parker said.
"Some of them would be measured in days, at the extreme," he added.
The Director-General of MI5 also highlighted how extremists can exploit "safe spaces" online, making the task of disrupting their activities more difficult.
Speaking in central London he issued a fresh challenge to technology firms, saying they have an "ethical responsibility" to help governments confront the threat.
He flagged up the ease of online purchasing, the use of social media platforms to broadcast "poisonous ideological messages" and the "myriad of ways in which encrypted communication can happen based on the thousands of different apps that offer those services".
Mr Parker said: "I believe there is a responsibility on the companies that offer those services to help governments be able to stop the worst excesses of human criminal behaviour."
The Director General did not single out any organisations, saying it would not "make sense" to identify individual firms.
He said: "For all the companies, and it's a wide field, there are many large companies and many more smaller ones, I think the same principle applies. There must be an ethical responsibility to help with these problems."
Mr Parker also noted that "some helpful action" is being taken over the issue.
Asked by ITV News if 2017 felt like a year of failure after five separate terror attacks were carried out, Mr Parker responded that the security service could not become caught up "in a situation where we're trying to imagine alternative realities. There are far too many variables involved in all these plots".
He continued that MI5 is "doing everything it possibly can" in a time when it is facing "more plots than we've seen before", and simply "cannot stop everything".
When pressed by ITV News Security Editor Rohit Kachroo that a number of attackers were known to the security services before they carried out their deadly plots, Mr Parker stated that "it's simply not possible in a free society to be across everything all the time," simply because counter-terrorism officials cannot "be inside people's heads".
London Bridge attack ringleader Khuram Butt was known to police and MI5 prior to the attack which killed eight people and injured 47 others, after he publicly associated himself with jailed hate preacher Anjem Choudary and other extremists in his Islamist network.
Despite the seemingly high number of terror attacks in 2017, a further seven have been foiled by M15 and the police this year, Mr Parker revealed.
He continued that in the last four years, 20 attacks have been stopped "and probably many more... stopped by the early interventions... made as soon as we see any signs of extremism and activity developing".
While the security service "cannot stop everything... it doesn't make it [feel] any less sharp when an attack does happen and people are killed", Mr Parker added.
Despite "knowing intellectually that we can't stop everything", MI5 staff do not become "inured" to attacks, instead "feeling it particularly strongly" when they are carried out, he said.
Mr Parker continued that he and his staff find it particularly "shocking and upsetting... because it's our profession to help stop those sorts of things happening."
It is this shock and upset that counter-terrorism workers use to motivate themselves, he explained.
"We put that energy into our operational work to be even stronger to do something about it," Mr Parker explained, adding that the only thing the security services can do in the wake of terror attacks is to "learn" from them.
"We're squeezing every single lesson we can," Mr Parker continued.
He revealed that there are still reviews underway into the terror attacks carried out in 2017, and "using the harsh light of hindsight... to look at all of the situation, every little bit of circumstance" to find out "what could we have known? What was there? And what learning can we take?"
A total of 36 people were killed in terror attacks on Westminster Bridge, London Bridge, Manchester Arena and at Finsbury Park in 2017, and countless others have had their lives changed by the attacks.
A further attack was carried out on a London Underground train at Parsons Green tube station in September, and while no one was killed, 30 people were injured.
Mr Parker continued that what MI5 "can learn" from past attacks "can make us just that bit stronger and sharper for the future".
He explained that the threat from terrorism is becoming a "more difficult situation" as more people "engage in extremism as the idea of violence becomes a bit more normalised".
"We have to keep pushing our work to stay ahead of the threat," he said, constantly "changing the way we work" to counter attacks in the future.
"We can't know everything," the head of MI5 conceded, "but we're doing everything we possibly can."