Working with EBS

Lab Overview

EBS

This lab focuses on Amazon Elastic Block Store (Amazon EBS), a key underlying storage mechanism for Amazon EC2 instances. In this lab, we will learn how to create an Amazon EBS volume, attach it to an instance, apply a file system to the volume, and then take a snapshot backup.

Topics covered

By the end of this lab, we will be able to:

Create an Amazon EBS volume
Attach and mount your volume to an EC2 instance
Create a snapshot of your volume
Create a new volume from your snapshot
Attach and mount the new volume to your EC2 instance

Let's get started.

What is Amazon Elastic Block Store?

Amazon Elastic Block Store (Amazon EBS) offers persistent storage for Amazon EC2 instances. Amazon EBS volumes are network-attached and persist independently from the life of an instance. Amazon EBS volumes are highly available, highly reliable volumes that can be leveraged as an Amazon EC2 instance boot partition or attached to a running Amazon EC2 instance as a standard block device.
When used as a boot partition, Amazon EC2 instances can be stopped and subsequently restarted, enabling you to pay only for the storage resources used while maintaining your instance's state. Amazon EBS volumes offer greatly improved durability over local Amazon EC2 instance stores because Amazon EBS volumes are automatically replicated on the backend (in a single Availability Zone).

For those wanting even more durability, Amazon EBS provides the ability to create point-in-time consistent snapshots of your volumes that are then stored in Amazon Simple Storage Service (Amazon S3) and automatically replicated across multiple Availability Zones. These snapshots can be used as the starting point for new Amazon EBS volumes and can protect your data for long-term durability. You can also easily share these snapshots with co-workers and other AWS developers.

Amazon EBS Volume Features

Amazon EBS volumes deliver the following features:

Persistent storage: Volume lifetime is independent of any particular Amazon EC2 instance.
General purpose: Amazon EBS volumes are raw, unformatted block devices that can be used from any operating system.
High performance: Amazon EBS volumes are equal to or better than local Amazon EC2 drives.
High reliability: Amazon EBS volumes have built-in redundancy within an Availability Zone.
Designed for resiliency: The AFR (Annual Failure Rate) of Amazon EBS is between 0.1% and 1%.
Variable size: Volume sizes range from 1 GB to 16 TB.
Easy to use: Amazon EBS volumes can be easily created, attached, backed up, restored, and deleted.

To follow the following tasks, we have to have an EC2 instance.

To create an instance, make the following configurations:

Provide your EC2 name
On Quick Start, Ubuntu for AMI
Instance type as t2.micro
Keypair: vockey
Network settings as default.
Launch Instance.

Let's continue.

Task 1: Create a New EBS Volume

In this task, we will create and attach an Amazon EBS volume to a new Amazon EC2 instance.

In the AWS Management Console, on the Services menu, click EC2.
In the left navigation pane, choose Instances.
Note the Availability Zone of the instance. It will look similar to us-east-1d (can be of any AZ).
Select your instance and Click on the Storage section to see the volume that is being used by the Amazon EC2 instance.

In the EC2 dashboard, choose Volumes.
Choose Create volume then configure:
- Volume Type: General Purpose SSD (gp2)
- Size (GiB): 1.
- Availability Zone: Select the same availability zone as your EC2 instance.
- Choose Add Tag
- In the Tag Editor, enter:
  - Key: Name
  - Value: My Volume
Choose Create Volume

Our new volume will appear in the list and will move from the Creating state to the Available state.

You may need to choose refresh to see your new volume.

Task 2: Attach the Volume to an Instance

You can now attach your new volume to the Amazon EC2 instance.

Select My Volume.
In the Actions menu, choose Attach volume.
Choose the Instance field, then select your instance.
Choose Attach volume The volume state is now In-use.

Task 3: Connect to your EC2 instance

Select your instance and click on Connect.

We're into our instance now.

Task 4: Create and Configure Your File System

In this task, you will add the new volume to a Linux instance as an ext3 file system under the /mnt/data-store mount point.

View the storage available on your instance:

df -h

You should see output similar to:

This shows the original 8GB disk volume. Your new volume is not yet shown.

Type the following command:

sudo lsblk -f

sudo: This is a command that allows you to execute another command with superuser privileges, essentially running the following command with administrative rights.

lsblk: This is a command used to list information about block devices, which typically include hard drives, SSDs, USB drives, and other storage devices. It provides details about the devices and their partitions.

-f: This is an option or flag for the lsblk command. The -f flag instructs lsblk to display additional information about the filesystems on the block devices. This can include information such as the filesystem type (e.g., ext4, ntfs), UUIDs, and labels.

When you run sudo lsblk -f, you'll get a listing of the block devices on your system along with information about the filesystems they contain.

"xvda" and "xvdf" are typically referred to as block devices in a Linux system, and they represent storage devices or disks. The specific naming convention "xvda" and "xvdf" is often used in virtualized environments, and the letters "xvd" are followed by a letter to represent different block devices. For example, "xvda" is commonly used for the root disk, and additional devices can be named "xvdb," "xvdc," and so on, for additional storage volumes or disks. These devices are used for data storage and can be partitioned and formatted with different file systems to store and manage data.

We have "xvda" as root disk while "xvdf" as the newly added disk that has no other aspects mentioned like type of file system and other additional infos.

Note the last section.

Now, let's create file system on our new disk.

sudo mkfs -t ext3 /dev/xvdf

mkfs: This is a command used to create a filesystem on a storage device. In your case, it's being used to create an ext3 filesystem.

-t ext3: This part of the command specifies the type of filesystem to create. In this case, it's set to "ext3." Ext3 (Extended 3) is a widely used Linux filesystem format known for its reliability and compatibility. It's considered a journaled filesystem, which helps with data consistency and recovery in case of unexpected system crashes or power failures.

/dev/xvdf: This is the target device on which the filesystem will be created. In Unix-like systems, block devices are typically represented as files under the /dev directory. /dev/xvdf represents a specific block device, and the command is instructing mkfs to create an ext3 filesystem on this device.

lsblk -f

Now, we have created the file system.

Create a directory for mounting the new storage volume:

sudo mkdir /mnt/data-store

Mount the new volume:

sudo mount /dev/xvdf /mnt/data-store

mount: This is the command used to mount a filesystem. When you mount a filesystem, you are essentially attaching it to a directory in the filesystem hierarchy, making its contents accessible at that location.

/dev/xvdf: This is the source device that you want to mount. In this case, you are specifying the /dev/xvdf block device as the source.

/mnt/data-store: This is the target directory where you want to mount the filesystem. The /mnt/data-store directory is typically an empty directory where you want the contents of the /dev/xvdf device to be accessible.

When you run this command with sudo, it will mount the filesystem from the /dev/xvdf device onto the /mnt/data-store directory. This means that the files and directories within the /dev/xvdf filesystem will become accessible under the /mnt/data-store directory.

To configure the Linux instance to mount this volume whenever the instance is started, you will need to add a line to /etc/fstab.


echo "/dev/xvdf   /mnt/data-store ext3 defaults,noatime 1 2" | sudo tee -a /etc/fstab

echo "/dev/xvdf /mnt/data-store ext3 defaults,noatime 1 2": This part of the command uses the echo command to create a text string, which is the entry that you want to add to the /etc/fstab file. This entry contains the following information:

/dev/xvdf: This is the source device you want to mount, which is the block device /dev/xvdf.
/mnt/data-store: This is the target directory where you want to mount the device.
ext3: This specifies the filesystem type as ext3.
defaults,noatime: These are mount options. "defaults" usually includes a standard set of options for mounting.
- In a Unix/Linux system, each file has three timestamps associated with it:
  1. Access Time (atime): This timestamp records the last time the file was accessed or read. Whenever you read a file, its atime gets updated.
  2. Modification Time (mtime): This timestamp records the last time the file's content was modified. When you write or change the content of a file, its mtime gets updated.
  3. Change Time (ctime): This timestamp records the last time the file's metadata (permissions, ownership, etc.) was changed. Even if you only change the file's permissions, ctime gets updated.
- The "noatime" mount option, as mentioned earlier, disables the automatic update of the access time (atime) when a file is read. This means that if you have "noatime" as a mount option, accessing a file won't cause the system to update its access time. Instead, the atime remains unchanged.
1: This is the dump frequency, indicating whether the filesystem should be included in the backup. A value of 1 means it should be included in backups.
2: This is the pass number used by the fsck utility to determine the order in which filesystems are checked at boot. A value of 2 means that the filesystem should be checked after the root filesystem (typically set to 1).
- The "pass number" in the /etc/fstab file is used to specify the order in which filesystems are checked by the fsck (file system consistency check) utility during the system's boot process. The fsck utility is responsible for verifying and repairing the integrity of filesystems to ensure they are consistent and error-free. This check is essential to prevent data corruption and maintain the overall stability of the system.
  
  The fsck utility performs its checks and repairs based on the pass number assigned to each filesystem in /etc/fstab. Here's how it works:
  - Pass Number 1 (typically set for the root filesystem, e.g., /): This is the first filesystem to be checked. In most cases, the root filesystem is assigned a pass number of 1 to ensure it is checked first during the boot process.
  - Pass Number 2 (and higher): Filesystems with pass numbers greater than 1 are checked after the root filesystem. By assigning a higher pass number, you can control the order in which non-root filesystems are checked.
|: This is a pipe operator that takes the output from the echo command on the left side and passes it as input to the command on the right side.

sudo tee -a /etc/fstab:
tee: The tee command reads from standard input and writes to standard output and files simultaneously. In this case, it's used to write data to the /etc/fstab file.
-a: This is an option for tee that stands for "append." It tells tee to append the data to the specified file, rather than overwriting its contents.
/etc/fstab: This is the path to the file you want to write to. In this case, it's the /etc/fstab file, which is a configuration file for mounting filesystems at boot.

So, when you run sudo tee -a /etc/fstab, you are appending data to the /etc/fstab file with superuser privileges.

echo "/dev/xvdf /mnt/data-store ext3 defaults,noatime 1 2" | sudo tee -a /etc/fstab

When you run this command, it will append the specified entry to the /etc/fstab file, which means that the system will automatically mount the /dev/xvdf device to /mnt/data-store with the specified mount options at boot time. This is useful for automating the mounting process and ensuring that it persists across reboots.

View the configuration file to see the setting on the last line:

cat /etc/fstab

View the available storage again:

df -h

On your mounted volume, create a file and add some text to it.

sudo sh -c "echo some text has been written > /mnt/data-store/file.txt"

sh -c: This part of the command is launching a new shell (sh) with the -c option, which tells the shell to execute the following command. In this case, it's used to execute the echo command.

"echo some text has been written > /mnt/data-store/file.txt": This is the command that is executed within the new shell.

echo: This is a command in Unix-like operating systems that is used to print a message or text to the standard output (usually the terminal).

some text has been written: This is the text that you want to print or write.

> /mnt/data-store/file.txt: The > symbol is used to redirect the output of the echo command to a file. In this case, it is redirecting the text to a file located at /mnt/data-store/file.txt. If the file doesn't exist, it will be created; if it does exist, its contents will be overwritten with the new text.

So, the overall purpose of this command is to use superuser privileges to write the text "some text has been written" to the file /mnt/data-store/file.txt.

Verify that the text has been written to your volume.

cat /mnt/data-store/file.txt

Task 5: Create an Amazon EBS Snapshot

In this task, you will create a snapshot of your EBS volume.

You can create any number of point-in-time, consistent snapshots from Amazon EBS volumes at any time. Amazon EBS snapshots are stored in Amazon S3 with high durability. New Amazon EBS volumes can be created out of snapshots for cloning or restoring backups. Amazon EBS snapshots can also be easily shared among AWS users or copied over AWS regions.

Click on Volumes and check out My Volume.

In the Actions, select on Create Snapshot.

In the tag section,

Key: Name

Value: My Snapshot

Click on Create Snapshot.

In the left navigation pane, choose Snapshots.

Your snapshot is displayed. The status will first have a state of Pending, which means that the snapshot is being created. It will then change to a state of Completed.

Note: Only used storage blocks are copied to snapshots, so empty blocks do not occupy any snapshot storage space.

Delete the file that you created on your volume.

sudo rm /mnt/data-store/file.txt

Verify that the file has been deleted.

ls /mnt/data-store/

Task 6: Restore the Amazon EBS Snapshot

If you ever wish to retrieve data stored in a snapshot, you can Restore the snapshot to a new EBS volume.

Create a Volume Using Your Snapshot

In the AWS Management Console, select My Snapshot.

In the Actions menu, select Create volume from snapshot.

For Availability Zone Select the same availability zone that you used earlier.

Choose Add tag then configure:

Key: Name
Value: Restored Volume
Choose Create volume

Note: When restoring a snapshot to a new volume, you can also modify the configuration, such as changing the volume type, size or Availability Zone.

Attach the Restored Volume to Your EC2 Instance

In the left navigation pane, choose Volumes.

Select Restored Volume.

In the Actions menu, select Attach volume.

Choose the Instance field, then select your instance.

Note that the Device field is set to /dev/sdg.

Choose Attach volume

The volume state is now in-use.

We can see the latest volume in the last row.

Mount the Restored Volume

Create a directory for mounting the new storage volume:

sudo mkdir /mnt/data-store2

Mount the new volume:

sudo mount /dev/xvdg /mnt/data-store2

Verify that the volume you mounted has the file that you created earlier.

ls /mnt/data-store2/

You should see file.txt.

Conclusion

We now have successfully:

Created an Amazon EBS volume
Attached the volume to an EC2 instance
Created a file system on the volume
Added a file to volume
Created a snapshot of your volume
Created a new volume from the snapshot
Attached and mounted the new volume to your EC2 instance
Verified that the file you created earlier was on the newly created volume

This concludes the lab.

Thank you.