We all know that user education and security awareness as a whole is broken in its current state. What is it that we can do to strengthen our weakest link, people? How can we demonstrate with the right type of metrics that we are successfully implementing change and producing a more secure line of defense? We treat information security defense as a process, and we preach defense in depth. There is a large portion of the information security industry that is focused on perimeter security. However, we are beginning to see a shift from strictly the data level protection to an increase in user level security and reporting. The security as a process and defense in depth mentality must be filtered down and implemented into our user training.
“The reason that most Security Awareness Training programs fail is because they are TRAININGS…. not Education.”
Experience and time in the industry shows that the Computer Based Trainings (CBTs) organizations require their employees to complete annually (or sometimes more often) are comparable to a compliance check box. It is a broken process. The employee is required to complete and pass this training for continued employment. Once the process is complete the knowledge is either forgotten or greatly reduced. One of the largest proven gaps occurs when the end user does not bring the information forward into their day to day working lives like they should. That is a large disconnect where it means the most. This is known as the Ebbinhaus Forgetting Curve. Repetition based on active recall has been demonstrated as effective in other areas for avoiding the curve and, therefore, is the foundational design such awareness programs should be based on.
“...basic training in mnemonic techniques can help overcome those differences in part. He asserted that the best methods for increasing the strength of memory are:
- better memory representation (e.g. with mnemonic techniques)
- repetition based on active recall (esp. spaced repetition).”
Bridging the Gap
Repetition is a proven successful way to bridge the gap of compliance, teaching our users real life skills, and helping secure the infrastructure that we are responsible for protecting. This is best implemented with a comprehensive hands-on security phishing and awareness rewards program. A full program design will provide a maturity that the CBTs have not. While they are a good value add and can be used to reinforce the real life scenarios, relying on them as a primary means of security awareness training will not provide the value or insight to the first line of defense. By consistently reinforcing the CBTs with a custom built awareness program you increase the end user’s skills and boost the organization’s immunity to phishing and social engineering threat factors.
Building Your Own Program
Building a mature and strategic program from the ground up is achievable with executive support and cultural alignment. An awareness program need not equate to thousands of dollars spent on creating flashy presentations and brown bag luncheons to draw crowds. Teaching by example and rewarding for good behavior is what will improve upon the user’s awareness.
“The point has never been to make everyone experts in security, it has always been to arm the employees with basic knowledge so that in the event something out of the ordinary occurs, it may help notify the security team.” 
An important takeaway and key point to remember is that it is not the employee’s responsibility to know the difference between a legitimate phish and spam, or that they should be hovering over links in emails before clicking. It is our job to have a program that is open enough and easy enough for them to report abnormalities or when something is not quite right.
1. Establish Objectives
The direction of an organization’s security awareness program should be tailor fit and reassessed periodically. With the constant changing threat landscape, maturity of user understanding, and a progressing industry, the objectives should be thought of as a moving target. An objective one year of decreased malware removals on desktops may mature past that to increased reporting of phishing/vishing attacks. However, establishing an aggressive set of objectives can result in a failed or unrealistic program. Concentrating on one or two achievable objectives at the beginning of a new program will allow you to accomplish a more specific goal. We can then adjust the target periodically to reflect the organization’s and program’s maturity.
2. Establish Baselines
Many organizations do not have formal security awareness training, so establishing a baseline should begin with a live fire exercise testing the skills and real world knowledge of a good subset of your users. Having a realistic outlook on where your security posture stands in relation to not only technical baselines, but also cultural norms should be standard practice. It is important to know how the users currently respond to threats and irregularities. Establishing an engagement with a certified and skilled penetration testing company can help you baseline these responses. By having a third party assess the skills of your users with professional phishing campaigns you will gain valuable insight into data that you may currently not have.
3. Scope and Create Program Rules and Guidelines
When the user or employee is being treated essentially as a customer, rules and guidelines should be well thought out and strategized. Miscommunications will only impede the learning process, making succeeding with the program more difficult. Align the rules to be consistent with the organization’s culture to have a higher adoption rate. Having multiple levels of input will enable you to have clear and concise program instructions and rules leading to an easier implementation.
4. Implement and Document Program Infrastructure
You are taught in driver’s education to wear your seat belt, look both ways, and adjust your mirrors. The first time you have a close call or even worse a real accident, you now have a real world experience that your mind falls back on each time you make a decision. It is the same with security awareness. The shock of the accident now gives the employee pause when future emails show up that may look a little odd and out of place. Afterwards the training teaches them what could possibly be at risk when they click through the illegitimate link. Setting up the phishing attacks to automatically redirect to a website that aligns with the program theme will create a connection between real life events and the message being presented for education.
5. Positive Reinforcement
One of the most important parts is letting them know that it is ok that they fell victim to the attack. This must be a consistent message throughout the education material. The more comfortable the user feels reporting the incident, the more cooperation and adoption you will witness. Assure the user that it will always be better coming from an internal training attempt than a real phishing attack, and practice makes perfect. The training should include what to look for, and more importantly how to report something abnormal. With a great first line of defense and solid Incident Response (IR) procedures, you will be far better off securing the human element, the weakest security link.
Gamification is actually a scientific term that roughly means applying game principles to a situation. The simplest definition of those principles is: 1) Goal establishment, 2) Rules, 3) Feedback, and 4) Participation is voluntary.
Being able to reward for good behavior is an essential part of the program as well. Employees should not feel ashamed to come to the right people for help, or afraid of being reprimanded for making a mistake. Gamification works well in many aspects of life, why should this be any different? Turn the program into something catchy and a small budget cannot just satisfy your expectations, but exceed them. Making a lottery of gift cards, discounted services, and other items to enforce the brand of the program and put something in the user’s hand will reinforce the message you are giving.
7 . Define Incident Response Processes
Incident response (IR) looks different in every organization. If you have a current proven method of IR you are already well on your way to including an awareness program into your current structure. Use the newly created program as a case study for testing procedures and policies. This will allow you to flush out any inconsistencies, inefficiencies, or unplanned situations. Assessing each step of the process will give the needed information to add or change policies to fit the needs of the organization around certain types of attacks.
Gaining Meaningful Metrics
“Successful metrics programs include well-defined measurements and the necessary steps to obtain them” 
There are an abundance of measurements to take throughout a security awareness program. Depending on your program and your goals you may have more tailor fit measurements to take.
Here are some common totals to track.
- E-mails sent
- Emails opened
- Links clicked
- Credentials harvested
- Reports of phishing attempts
- Emails not reported on
- Hits on training sites
Tracking success rate and progress
Keeping track of click percentages, phishes reported, and incidents reported is a good start and necessary. However, charting your gains and losses with structured data over time will give your organization a deeper understanding of the progress made. Successful education and retained knowledge will be apparent with the increase and decrease of certain measurements and the success of goals set for metrics. Periodic assessment of shifts in metrics should be performed to assist with guidance of the education program’s goals and other possible implementations or changes in the current environment's security structure.
Measures are concrete, usually measure one thing, and are quantitative in nature (e.g. I have five apples). Metrics describe a quality and require a measurement baseline (I have five more apples than I did yesterday).
The metric of how much your security posture has increased in reference to your baseline is the key goal and quality control. Seeing increased reporting changes in suspicious activity on your network should align with a lower amount of malware, DNS queries to blocked sites, or other activity on the network that would lead an analyst to believe the possibility of a targeted attack has been blocked. The ability to link key metrics back to specific departments, buildings, or roles provides the information you need to scope more directed education.
- Building an Information Security Awareness Program: Defending Against Social Engineering and Technical Threats - Bill Gardner & Valerie Thomas