Security and Protection in Operating System
Subject: Computer Science Fundamentals · Language: Operating System · Level: advanced · 9 min read
Learn OS security and protection, authentication, authorization, access control, threats, encryption, malware, and safe resource access.
Security and protection are operating system responsibilities that keep computer resources safe from unauthorized access, misuse, damage, and data loss. The OS manages users, programs, files, memory, devices, and network access, so it must also control who can use what. Security focuses on defending the system against threats such as attackers, malware, stolen passwords, and unauthorized access. Protection focuses on controlling access to system resources so processes and users do not interfere with each other.
For beginners, security and protection can be compared with a college campus. Security checks who is allowed to enter, while protection rules decide which rooms, labs, records, or equipment each person can use. The operating system uses similar checks for digital resources.
Why Security Is Needed
Computers store important data such as personal files, passwords, bank details, exam records, company documents, and application data. If this data is accessed or changed by the wrong person, serious harm can happen. Modern computers are also connected to networks and the internet. This increases the chance of attacks from outside sources. The operating system must protect data, keep programs under control, and prevent unauthorized users from damaging the system.
Why Protection Is Needed
Protection is needed because many users and processes may share the same computer resources. Without protection, one process could read another process's memory, delete another user's files, or take control of a device. Protection keeps each process within its allowed limits. It also helps the OS enforce rules such as file permissions, memory boundaries, and device access rights. A protected system is more stable because mistakes in one program are less likely to affect the whole machine.
Security vs Protection
Security is a broader concept. It deals with protecting the system from threats, attacks, unauthorized access, and data theft. Protection is more focused on internal control. It decides what each user or process is allowed to access after they are inside the system. Both are connected. A system cannot be secure if protection rules are weak, and protection rules are less useful if attackers can easily bypass login or system checks.
Authentication
Authentication means verifying the identity of a user or process. In simple words, the system asks: who are you? Common authentication methods include passwords, PINs, biometric checks, security tokens, smart cards, and one-time passwords. If authentication is weak, attackers may enter the system by guessing or stealing login details. Strong authentication reduces this risk.
Authorization
Authorization means deciding what an authenticated user or process is allowed to do. In simple words, the system asks: what are you allowed to access? For example, a student may be allowed to view their own result but not edit marks. An administrator may have permission to manage users and system settings. Authentication comes first, then authorization. Knowing who the user is helps the OS decide which permissions apply.
Access Control
Access control is the method used to enforce authorization rules. It controls access to files, directories, memory, devices, network ports, and other resources. A common access control model uses permissions such as read, write, and execute. Read allows viewing data, write allows changing data, and execute allows running a program. Access control prevents users and processes from performing actions outside their rights.
Access Control Matrix
An access control matrix is a conceptual table that shows which subjects can access which objects and what operations they can perform. Subjects may be users or processes. Objects may be files, devices, directories, or memory regions. Each cell in the matrix stores permissions. This model helps students understand protection clearly, although real operating systems may implement permissions using more efficient structures.
Access Control List
An Access Control List, or ACL, stores permissions with each object. For example, a file may store a list of users and the actions each user can perform on that file. ACLs are useful when different users need different permissions for the same file or resource. They are common in many operating systems because they provide flexible permission control.
Capability List
A capability list stores permissions with each subject. Instead of asking what users can do to one file, it asks what resources one user or process can access. A capability acts like a protected key or token that allows access to a resource. Capability-based protection is useful for understanding another way operating systems can organize access rights.
User Mode and Kernel Mode
Modern operating systems usually separate execution into user mode and kernel mode. Normal applications run in user mode, while the operating system kernel runs in kernel mode. User mode is restricted. Programs cannot directly access hardware or critical memory areas. If they need a service, they must request it through system calls. Kernel mode has higher privileges. This separation protects the system from faulty or malicious user programs.
Password Security
Passwords are a common authentication method, but they must be handled carefully. Weak passwords can be guessed, reused passwords can be stolen, and plain-text storage is unsafe. Operating systems should store password hashes instead of actual passwords. A hash is a one-way representation that makes it harder to recover the original password. Users should use strong and unique passwords, and systems may add extra protection such as account lockouts or multi-factor authentication.
Encryption
Encryption converts readable data into unreadable form using a key. Only someone with the correct key can convert it back to readable form. Encryption protects data stored on disks and data sent over networks. Even if someone steals encrypted data, they cannot easily understand it without the key. Operating systems may support disk encryption, file encryption, and secure communication features.
Malware
Malware means malicious software. It includes viruses, worms, trojans, ransomware, spyware, and other harmful programs. Malware may steal data, damage files, lock the system, spy on users, or spread to other computers. The operating system helps reduce malware risk through permissions, user mode restrictions, updates, antivirus support, and controlled execution policies.
Virus and Worm
A virus attaches itself to files or programs and spreads when those files or programs are executed. It usually needs some user or program action to spread. A worm can spread across networks by itself by using vulnerabilities. Worms can quickly affect many systems if they are not controlled. Both are harmful, but the way they spread is different. This difference is commonly asked in basic security discussions.
Trojan and Ransomware
A trojan appears to be useful or harmless software but secretly performs harmful actions. Users may install it because it looks legitimate. Ransomware locks or encrypts files and demands payment to restore access. It is dangerous because it directly attacks user data. Regular backups, careful downloading, updates, and security tools help reduce these risks.
System Threats
System threats target the operating system and its services. Examples include privilege escalation, denial-of-service attacks, unauthorized login attempts, and exploiting software vulnerabilities. A privilege escalation attack tries to gain higher permissions than allowed. A denial-of-service attack tries to overload a system so normal users cannot use it. The OS must monitor, restrict, and update system components to reduce these threats.
Audit and Logging
Audit and logging means recording important system events. Logs may include login attempts, file access, permission changes, errors, and security warnings. Logs help administrators detect suspicious activity and understand what happened after a problem occurs. Good logging is important, but logs must also be protected because attackers may try to delete or modify them.
Backup and Recovery
Backup means keeping copies of important data. Recovery means restoring data or system state after failure, attack, or accidental deletion. Backups are an important part of security because even strong protection cannot prevent every possible problem. Hardware can fail, users can make mistakes, and malware can damage files. A good backup plan keeps important data safe and allows the system to return to normal faster.
Principle of Least Privilege
The principle of least privilege says that a user or process should get only the permissions needed to complete its task, and nothing extra. This reduces damage if a program has a bug or gets attacked. If the program has limited permissions, it cannot easily harm unrelated files or system areas. Least privilege is one of the most important ideas in operating system security.
Summary
Security protects the system from threats, while protection controls access to resources. Operating systems use authentication, authorization, access control, user/kernel mode separation, encryption, logging, and permission rules to keep systems safe. For beginners, the key idea is that the OS is not only a manager of CPU, memory, and files. It is also a guard that checks identity, limits access, records important events, and reduces the damage caused by mistakes or attacks.
More Operating System Lessons
- Introduction to Operating System
- Types of Operating System
- Operating System Functions and Services
- System Calls in Operating System
- Process Management in Operating System
- Process Scheduling in Operating System
- CPU Scheduling Algorithms
- Threads and Multithreading in Operating System
- Process Synchronization in Operating System
- Deadlock in Operating System
- Memory Management in Operating System
- Virtual Memory in Operating System