Saturday, December 31, 2016

Virtualization & Cloud - Scratchpad

I usually spend last two weeks of an year to do a self assessment and finish some of the pending tasks. 

I decided to do the following:

- Finish reading the books in pipeline
- Improve my technical skills  (Virtualization & Cloud)

I'd like to say that I managed to accomplish both the above tasks.
Some of my notes from VMware virtualization study are given below. I will add some notes about my book reading summary soon.

VMware is complex compared to middleware and database. Setting up middleware and database is easy compared to VMware. vCenter itself has lot of configurations. You are essentially managing entire data center using vCenter.

If you want to claim that you are an expert in VMware and vCenter, you need to know the three technologies listed below:

1) Storage, 2) Virtualization and 3) Networking

Storage and Virtualization is tied together. It is better that all your VMware team members know these three technologies.

VMware offer free labs and it is a great way to know the products. Best way to learn this is not by reading; but by actually doing some of the tasks.

Example: use the free VMware labs to configure a ESXi server with clusters. Perform simple tasks as vMotion, configure HA etc.

Basics and Fundamentals: 


Virtualization is the creation of a virtual (rather than actual) version of something, such as an operating system, a server, a storage device or network resources.

Here we are talking about virtualization of hardware.

You can install software on top of hardware or virtualize hardware and use virtual machines for installing software.

You virtualize hardware by installing Hypervisor on the hardware. Hypervisor installation of hardware allows you to use it as virutual machines.

You can install hypervisor on a single hardware and use it for virtualization. How about installing hypervisor on multiple machines and managing it?

Think about a data center.

How do we manage an entire data center with hypervisor deployed in every hardware? You need a centralized management of all hypervisors and obtain the additional benefits such as vMotion , clustering, HA, replication etc.

That is why you need
vCenter - a centralized management for all hypervisors.

VMware hypervisor is called ESXi.

Software Defined Data center (SDDC) is where all infrastructure is virtualized. Virtualization allows you to run more workload on a single host. Target for utilization is 75%.


It is a piece of software that creates and run virtual machines. There are Type 1 and Type 2 hypervisor.

Type1 - Baremetal hypervisor runs on top of hardware .  Some examples are VMware ESXi, Xen, Microsoft Hyper-V, etc.

Type2 - operate as an application on top of operating system. Examples of this type of hypervisor include VMware Fusion, Oracle Virtual Box, Oracle VM for x86, Solaris Zones, Parallels and VMware Workstation.

VMware ESXi (formerly ESX) is a type-1 hypervisor developed by VMware for deploying and serving virtual computers. As a type-1 hypervisor, ESXi is not a software application that one installs in an operating system (OS); instead, it includes and integrates vital OS components, such as a kernel.

ESXi is not an operating system.

The name is derive from "Elastic Sky X".  It has a kernel - VMkernel.

The VMkernel is the kernel used by VMware ESX. It provides basic operating system services needed to support virtualization: hardware abstraction, hardware drivers, scheduler, memory allocator, filesystem (vmfs), and virtual machine monitor (vmm).

You don't need vCenter to access ESXi. You can directly manage ESXi using DCUI. 

The Direct Console User Interface (DCUI) is the front-end management system that allows for some basic configuration changes and troubleshooting options should the VMware ESXi host become unmanageable via conventional tools such as the vSphere Client or vCenter. 

Direct Console User Interface (DCUI) - enable administrator to configure root access settings. It is slimier to BIOS of a computer.  Restrict access to command line. Control root and control CLI.

VMware vSphere: 

This is server virtualization platform by VMware.

VMware vSphere versions available are 6.5, 6.0, 5.5, 5.1 and 5.0. Most of the organization goes through frequent upgrades. Upgrades are managed by two stages:

- vCenter upgrades
- ESXi upgrades

VMware vSphere includes the VMware ESX / ESXi hypervisor, a type 1 hypervisor (that functions as the virtualization server), the VMware vCenter Server (which manages vSphere environments), the VMware vSphere Client, (which is used to install and manage virtual machines through the hypervisor) and VMware VMFS (the file system).

It is available for down load as:

  • vSphere essentials - EsXI hypervisor & Vcenter
  • Essential plus - include replication (disaster recovery solution)
  • and data protection - disk-based backup and recovery solution
  • Standard - includes Log insight & Orchestration appliance.
  • Enterprise plus - includes Standard and Operations manager, Big Data extensions and integrated containers.
  • Desktop


vCenter Server is basically a management tool that will allow you to manage VM's across multiple VMware hosts at once. You'll connect to vCenter Server with avSphere client and instead of one host you'll see multiple hosts at once.

Vcenter has a database that keep track of all the changes. You can always login in to individual hosts and make changes, however vcenter may not be knowing about it.

You can still login to Hypervisor and manage the host and vms using DCUI, however vCenter don't know about these.
That creates problem for the data in vCenter.

vCenter is organized by Data center, Clusters, hosts (ESXI), Virtual machines.

Most of the things that you do with vCenter can be scripted by using Power shell or VCLI (vSphere Command-Line Interface)

VMware vcenter makes it easy for admins to manage infrastructure using:

  • Snapshot
  • Clone 
  • vapps (OVF Templates) 
  • Template 
  • Host profile 
  • Auto VM Vmotion
  • Storage VMotion
  • DataStore
  • VLAN
  • Distributed switch etc.

Network Fundamentals:

Your IP address is your computer's unique address on the Internet. Every Ethernet adapter has what's called a Media Access Control (MAC) address, also referred to as a hardware address or physical address.

Enter ipconfig on windows or ifconfig in our Linux or mac terminal  you will find your IP4 and IP6 address . 

Mac address is your machine address . It is the HARDWARE address of any device. This address will never change . It is the unique machine address given to your device. Your device will have communication with the local area network or any network using this address . 

Once you use the network, your mac address is stored in the network table so that network does not have to search your mac address again and again and the shortest path towards it will be stored in routing table . That's why never feel that no one will come to know if you use your neighbor's wifi, remember that router stores everything,  it's a smart and intelligent gateway.

Physical Ethernet adapter also called a "network interface card" (NIC), an Ethernet adapter is a card that plugs into a slot on the motherboard and enables a computer to access an Ethernet network (LAN). In the past, desktop computers always used cards, today it is using only in servers. 

A network interface controller (NIC, also known as a network interface card, network adapter, LAN adapter or physical network interface, and by similar terms) is a computer hardware component that connects a computer to a computer network.

Private VLAN, also known as port isolation, is a technique in computer networking where a VLAN contains switch ports that are restricted such that they can only communicate with a given "uplink". The restricted ports are called "private ports".

A virtual LAN (VLAN) is any broadcast domain that is partitioned and isolated in a computer network at the data link layer (OSI layer 2). LAN is an abbreviation for local area network. To subdivide a network into virtual LANs, one configures network equipment.

Net Flow is a network protocol developed by Cisco for collecting IP traffic information and monitoring network traffic. By analyzing flow data, a picture of network traffic flow and volume can be built.

Virtual Network:

A Virtual Switch Is Similar to a Physical Switch. A virtual switch, as implemented in ESX Server 3, works in much the same way as a modern Ethernet switch. It maintains a MAC:port forwarding table.

Distributed switch is a network switch directs network traffic. Similarly, a virtual switch (vSwitch) carries VMs' traffic to the physical network and to other VMs. Distributed virtual Switches, which are also known as VMware vDS, enable more features than standard vSwitches, sometimes called VMware vSS.

The vNetwork Distributed Switches (vDS) for short allows you to configure a single virtual switch to span multiple hosts,so you don't need to create virtual machine port groups on all your hosts.

vSwitch connect to kernel port. The VMkernel ports are special constructs used by the vSphere host to communicate with the outside world. It also referred to as “VMkernel networking interfaces” or “virtual adapters”. The VMkernel port group is created when an ESXi host is deployed. It is for communication of the ESXi host.

vLan (Virtual Lan) - virtual organization of Lan. if we do vlan then we can isolate traffic.

Switch provide communications between VMs - provide connections to virtual machines. Multiple uplinks can be added to the vSwitch and they are connected to the physical switch. 

Distributed switch vs. standard switch.

A network switch directs network traffic. Similarly, a virtual switch (vSwitch) carries VMs' traffic to the physical network and to other VMs. 

Distributed vSwitches (VDS) which are also known as enable more features than standard vSwitches (VSS). The only difference is VDS span across multiple hosts. 

One switch can have multiple port groups

I can have one switch for everything or separate switches for everything. Network can existing on separate virtual switches.

It is better to have one virtual switch instead of configuring multiple virtual switches. Each virtual switch required two NIC.  

vSS is This is a vSphere Standard Switch. Uplink is the physical NIC, which connects to an external physical network.

Network adapter - auto negotiate is the default. Ensure that the physical switches are set for auto negotiate.

Network Load balancing methods:
  • MAC hash
  • Round robin
  • Source and Destination IP Hash (It is a true load balancing method, based on session it will route traffic. It actually examine the traffic to determine with NIC it should send to.
Virtual Switch - Not a load balancer, it is load distributor., It is a round robin. This will avoid NIC 1 hitting all the time.

  • Virtual machine Vmotion - VMs (memory) move from one host to another host
  • Storage vmotion - The data store move 

There are many enhancements in vSphere 6.5, an overhaul of Storage IO Control is one of them. In vSphere 6.5 Storage IO Control has been reimplemented by leveraging the VAIO framework. For those who don't know VAIO stands for vSphere APIs for IO Filtering.

Storage vMotion is a component of VMware vSphere that allows the live migration of a running virtual machine's (VM) file system from one storage system to another, with no downtime for the VM or service disruption for end users.

Storage DRS allows you to manage the aggregated resources of a datastore cluster. When Storage DRS is enabled, it provides recommendations for virtual machine disk placement and migration to balance space and I/O resources across the datastores in the datastore cluster - Fix this going forward.

You can set Storage DRS thresholds

Virtual machine:

Host is physical hardware. VM is virtual machine. After installing hypervisor on a host you can install VM on it. After configuring the hypervisor with Vcenter you can manage it using centralized console.

Virtual machine is the virtual hardware. Similar to I can have a host(physical hardware).

Virtual machine don't need to have an OS on it!. I can create a VM and not installed an OS on it.

Maximum memory that can have for a VM are listed below:

Version 6.5 - maximum 6 TB
Version 6.0 - maximum 4 TB

Each virtual machine consists of set of related files. Configuration file - vmx,
Swap files - Vswap (allow us to over allocate memory).

Template file - vmtx. We can create other VMs from template file.

Virtual disk file - VMDK. VMDK (Virtual Machine Disk) is a file format that describes containers for virtual hard disk drives to be used in virtual machines like VMware Workstation or VirtualBox.

Two files by each VM. 1) vmname.vmdk 2) vmname-flat.vmdk. Access the flat vmdk file through raw device map file vmname-rdm.vmdk.

A Virtual machine consisits of the following files:
    • The -flat.vmdk fille
    • The -delta.vmdk file
    • The -rdm.vmdk file
    • The .vswp file
    • The .vmss file.
    • The .vmsd file
    • The .vmsn file (The memory state file of snapshot - .vmsn)
    • The .log file
    Suspend VM is paused VM. The state of VM is saved.The suspend and resume feature is useful when you want to save the current state of your virtual machine, and continue work later from the same state.

    Pausing a virtual machine stops the current state of a virtual machine. When you resume a paused virtual machine, the state of the virtual machine is exactly the same as when you paused it.

    Pause VM feature is useful when a virtual machine is engaged in a lengthy, processor-intensive activity that prevents you from using the host system to do a more immediate task.

    You suspend a virtual machine when you want to save its current state. When you resume the virtual machine, applications that were running before the virtual machine was suspended resume in their running state and their content is unchanged.

    The applications running top of suspended or Paused VM will not get impacted. It will keep running.

    CPU & Memory allocations

    Similar to OS need memory, Virtual machine itself need some memory for it's working.

    VSWP file size should be accounted for when you calculate memory:  
    Memory = Allocated memory - reserved memory. Please consider  active memory versus allocated memory, boot time behavior of the OS and paging etc. 

    CPU is always underutilized. Majority of the VMs in an organization don't use more than 10vCPU. Majority of the cases, memory is the bottleneck. Allocating too much Memory or too little can force contention. 
    For a a single VM, we can not have more vCPU than the logical CPU available.

    These files are created equal in size to the amount of memory assigned to a VM, minus any memory reservations -- default is 0 -- that a VM may have set on it (i.e., a 4 GB VM with a 1 GB reservation will have a 3 GB VSWP file created).

    SSD is waste.
    SSD is very expensive. It is better to buy more memory rather than purchasing SSD.

    It is OK to over allocate memory for VM.

    There can be bottleneck in CPU, Memory, Storage and Networking

    CPU - Three different levels of CPU are listed below. VMware refers to CPU as Physical CPU (pCPU) and and Virtual CPU (vCPU).

    1) vCPU
    2) Physical CPU (pCPu - Socket)
    3) lCPU Logical CPU (Core/thread - Logical CPU) = Socket X CORE per Socket

    To calculate CPU per cluster, you need to add the CPU of each of the hosts in that cluster.

    • A single core processor can have singe processor, thread.  Do single thing at a timer.  
    • Duel core can have  two things do at the same time. 
    • Multiple core provide ability to multiple things at the same time, single processor can do processing only one at a time.     
    • Hyper threading allow queues, it will allow us to do multiple things at the same time.  More people can order the items.
    2 duel core processor, how many vCPU? Answer is 64 (32 vCPU per processor). Once Core can have 32 vCPU. In vSphere 5.0 with Enterprise Plus Licensing, the maximum number of vCPUs per Virtual Machine is 32

    If you enable hyper threading, then we can have double vCPU.

    A processor can have multiple CORE.

    Physical socket on a motherboard where a physical processor fits
    Physical core within a physical processor. You can have multiple cores for a processor.
    Logical core within a physical core (hyper-threading)

    Resource pools is a logical abstraction of CPU and Memory. I can share this resource pool for multiple VMs.

    The reservation is relative. Set Memory Reservation on a Virtual Machine To avoid memory over commitment, you must reserve all of the memory allocated to each virtual machine that runs in the VSA cluster.

    If I have only 4 CPU, then one VM can allocate only 4 vCPU. If I turn hyper threading then I can can have more vCPU.

    Reservation is the memory that you are reserve it, limit is the limit that a VM can go and utilize. For CPU, there is no limit.

    We can prioritize the VMs by allocating shares. It is like stock. The VM that has more shares (relative) will get more resources.  

    VMM is responsible for scheduling all the resources.

    Log files:

    Every virtual machine has it's own log. Vmware.log. Each reboot will create separate vmware.log file.

    You can use log insight tool to configure view log files. Other option is to obtain the SUDO access for log files.
    Log files - Server log, vcenter Server agent log etc. Some of the log files are given below.

    SX/ESXi service log/var/log/vmware/hostd.log 
    vCenter Server agent log/var/log/vmware/vpx/vpxa.log
    Virtual machine kernel core file/root/vmkernel-core.
    syslogd log/var/log/messages

    Numa (non uniform) - NUMA systems are advanced server platforms with more than one system bus. They can harness large numbers of processors in a single system image with superior price to performance ratios.
    VMware VMFS (Virtual Machine File System) is a cluster file system that facilitates storage virtualization for multiple installations of VMware ESX Server, a hypervisor that partitions physical servers into multiple virtual machines. VMFS is part of the virtualization suite called VMware Infrastructure 3.

    Virtual Disk - A virtual machine will have at least one virtual disk.

    Thin provisioning will grab the storage only when it is needed. Thick provisioning immediately grab the storage irrespective of the usage.

    Virtual Network

    Network adapters (NIC). simier to vCPU there is vNiC. E-1000 - E1000E is high performance adaptor avaialble for only some of the guest operating system.

    VMXNET and VMNET2, VMXNET 3 are vmware drivers that are available only for VMware tools.

    Virtual machine components - OS, VMware Tools, Virtual resources such as CPU, Memory, Network adaptors, Disk controllers, parallel and serial ports.

    Virtual machine is nothing but a file (VMDK). VMs are easy to relocate from one host to another host. It allow us to consolidate hardware

    Virtual Machine Manager (VMM) - A scheduler for virtual resources. VMM talk to all Virtual Machines.

    Network resource pool is distributed switch.

    Vmware vcenter

    Vmware Vcenter is a centralized management application that manage virtual machines and ESXi hosts. vSphere client is used to access the vcenter. in 6.5, it is replacement with web browser based console.

    Vcenter has a database. You can re-install vcenter and point to database it will pull the data.

    platform service controller - Certificate authority, single sign on etc.

    Vcenter consists of Database server, platform Services controller, CORE services, eSXi management, User access control and vSphere API

    Directory server (identity sources) will have all access controller.

    Vcenter administrator

    • Single sign on administrator - Vcenter and Single sign on server. He can configure any identity services.
    • For others - no access. We need to create roles and provide access for others to access vcenter.

    Self signed certificate or third party trusted certificates.

    esxi will have vpxa agent that will be enable to talk to vcenter.VPXA is the agent of Vcenter server.

    vcenter server will have vpxd to communicate back to esXI hosts.
    VPXD is Vcenter Server Service. If this service is stopped then we will not able to connect to Vcenter Server via Vsphere client.  

    hostd is an app that runs in the Service Console that is responsible for managing most of the operations on the ESX machine. HOSTD is is the agent of ESX server, here VPXA pass the information to the HOSTD and hostd pass the information to ESX server.

    Do not mess with vpxd user and password !

    Once the ESXI host is configured with vcenter, never login in to the Vcenter Server. Ideally any changes to ESXI host should be stored in Vcenter database.

    Locked down mode will prevent administrators to login to ESXi hosts. Incase Vcenter is down, you can go directly to ESXI host and do whatever you want. Onc scenario is you enabled locked down mode, and lost venter, you need to use exceptional users to login to esxi host and make changes.

    platform service has all information all access and autherization.

    • vcenter 6.5 has limit of 1000 hosts and 35000 VM limit
    • Vcenter appliance is SUSE Linux
    Use appliance version for install. From 5.5 on wards, appliance seems to be stablilized. Easy to manage. Use windows only if you want to control the OS such as patches etc. For appliance, you need to wait for VMware to release the patches.

    Naming convention is very important. Do not name the datacenter as data center 1,2,3 etc! You can create custom tags for inventory objects.

    Vmotion Limitations

    • You can't vmotion of vms across data centers
    • You can't create distributed switches across data centers. 
    • You can vmotion VMs between Vcenters, but not across data centers within one vcenter.
    • You can't vmotion between chipset without shutting down vm. no live vmotion available between chipsets.  

    If storage is slow then every thing else is slow. We always need high performance storage for better performance.  

    Lun - Logic unit number.

    Storage pool is a collection of physical disks. once you assign the storage pool, you assign a Raid level

    How to present these LUN to others.

    Two types of Lun

    Thin - Initial size is zero. We grab the space as needed. Host see 100 G, however storage array see zero. The issue with thin provisioning is the VM may through a disk full error.  The write may fail.

    Over provision up to 120-130% of available space. not more that.

    Be very careful for over provisioning using "thin" provisioning mechanism. There could be some catastrophic failure due to non-availability of storage. Always monitor the storage capacity when you provisioning

    Thick or Thin?

    Which storage type is better? it is like which is better? google or yahoo? The question is what is better for you? Can you afford fiber. Iscsi will be fine for most of the case. You just need to make sure that you have enough back bone.

    IScSI - cheaper
    Fiber - Faster. The price come down a lot recently.

    FCoE (Fibre Channel over Ethernet) is the newest storage networking protocol of the bunch. Ratified as a standard in June of last year, FCoE is the Fibre Channel community's answer to the benefits of iSCSI. Like iSCSI, FCoE uses standard multipurpose Ethernet networks to connect servers with storage

    iSCSI, which stands for Internet Small Computer System Interface, works on top of the Transport Control Protocol (TCP) and allows the SCSI command to be sent end-to-end over local-area networks (LANs), wide-area networks (WANs) or the Internet.

    Raw device map - Lun is knows as device in VMware world. Two format VMFS3 or VMFS5.

    VMware VMFS (Virtual Machine File System) is a cluster file system that facilitates storage virtualization for multiple installations of VMware ESX Server, a hypervisor that partitions physical servers into multiple virtual machines. VMFS is part of the virtualization suite called VMware Infrastructure 3.

    Maximum size of data store is 64 TB. Maximum size of VMDK 62 TB

    • Eager zero
    • Lazy zero
    • Thin
    For VMs, you may need to thin VMs on thick storage provisioning. Do not do thin on thin because fragmentation cause the problem.

    You need to consider the following whenever you take these decisions.
    Lock on contention

    I can't de-fragment the VMDK

    VMDK (Virtual Machine Disk) is a file format that describes containers for virtual hard disk drives to be used in virtual machines like VMware Workstation or VirtualBox.

    Types of storage:
    Fibre Channel,  FCoE, iSCSI , NFS , DAS, Virtual Volumes ,Virtual SAN

    Whether storage is provided through block-based (iSCSI, Fibre Channel, FCoE) or file-based (NFS) protocols, vSphere virtual machines are all stored in a logical object known as a datastore.

    Virtual Volumes (VVOLs) are something new that VMware is working on to improve storage provisioning. Storage currently tends to be provisioned according to a gold, silver and bronze type of model that forces a virtualization administrator to pick the storage tier that most closely matches their needs

    VVOLs allow policy-based metrics to be applied to storage for an individual virtual machine rather than at the datastore level

    Storage I/O control  (Data store latency) - any thing above 10ms is bad. Shares are irrelevant unless there is some kind of contention.

    DRS - Automated vmotion of VMs based on the host capacity

    Storage DRS - Automated vmotion of Data store based on data stores capacity

    Should not cluster data stores with different arrays. Storage vmotion will take so much of time.

    backed up array with non-backed up arrays.You can have affinity rules DRS, you can have affinity rules with Storage DRS for storage.

    Storage I/O control - fix it now for the issue related contention
    Storage DRS - Will solve for storage contention later.

    CHAP: iSCSI initiatros use CHAP for  security (Authentication) purpose.

    A data store is a repository for persistently storing and managing collections of data which include not just repositories like databases, but also simpler store types such as simple files, emails etc. A database is a series of bytes that is managed by a database management system (DBMS).

    A datastore is platform-independent and host-independent. Therefore, datastores do not change when the virtual machines they contain are moved between hosts. The scope of a datastore is a datacenter; the datastore is uniquely named within the datacenter.

    A datastore, as it is used by VMware, is a storage container for files. The datastore could be on a local server hard drive or across the network on a FC or iSCSI SAN. Inside the datastore, you will find the virtual machines, VM disks, VM configurations, and any other files you place in the datastore (such as ISO files to install a VM). Datastores are using in VMware ESX Server (viewed with the VI Client) and in the new VMware Server 2.0.

    Allows to connect to NFS protocol Version3 or Version 4.1 (can have multiple path going to the NFS data store)
    VMstorage policy can be VVOL or Virtual SAN.

    Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems in 1984, allowing a user on a client computer to access files over a computer network much like local storage is accessed.

    The VMkernel ports, which are also referred to as “VMkernel networking interfaces” or even “virtual adapters” in various places, are special constructs used by the vSphere host to communicate with the outside world:
    We won't be able to IP storage without VMkernal port.

    Create NFS Datastore in the vSphere Client
    You can use the Add Storage wizard to mount an NFS volume and use it as if it were a VMFS datastore

    VMFS deposits files in a storage container called a datastore, a separate filesystem that runs on top of a volume. Datastores reside on a block-based device such as iSCSI. Virtual machine disks (VMDKs) are housed in the datastore.

    Host to NFS3/NFS4.1/VMFS - Data store - Lun - Storage arrays.

    VSAN - Software defined storage by VMware (Other option is VVOL)

    Virtual SAN (VSAN), VMware’s software-defined storage for vSphere. It combines local server disks in ESXi hosts to create shared storage. 

    This shared storage can be accessed from any hosts. 

    Vsan is only for internal storage (supports SAS/SATA/SSD/HDD ). For external enterprise storage you still use traditional storage, or you can use volves

    VSAN being a software defined storage solution gives the customer the much needed flexibility where you are no longer tied in to a particular SAN vendor.
    You no longer have to buy expensive EMC or NetApp disk shelves either as you can go procure commodity hardware to design your DC environment as you see fit

    In VSAN, we can create storage policies. VSAN storage all have storage policies.

    Since VSAN drivers are built in to the ESXi kernel itself (Hypervisor), its directly in the IO path of VM’s which gives it superior performance with sub millisecond latency

    Also tightly integration with other beloved vSphere features such as VMotion, HA, DRS and SVMotion as well as other VMware Software Defined Datacenter products such as vRealize Automation and vSphere replication.


    The virtual machines are defined by files.

    Automation Vmotion should be within a cluster, however you can vmotion VMs anywhere.

    When vomotion, the actual memory is getting tranferred from one host to another host.

    You can't pin the Vm to a local resource. This will prevent vmotion.  You need a shared storage. Atleast 1GigE network. 10G is recommended for  vmware.

    Any local mount will prevent Vmotion. For example, if you connect to CD Drive, or RDM then Vmotion won;t happed. You should unmount your CD before vmotion.

    RDM is a mapping file in a separate VMFS volume that acts as a proxy for a raw physical storage device. The RDM allows a virtual machine to directly access and use the storage device. The RDM contains metadata for managing and redirecting disk access to the physical device.

    Chipset family is restriction for vmotion.

    DRS (Distributed Resource Scheduler)

    With VMware DRS, users define the rules for allocation of physical resources among virtual machines. The utility can be configured for manual or automatic control. Resource pools can be easily added, removed or reorganized. If desired, resource pools can be isolated between different business units.

    How to ensure that Vmotion works?

    Do the vmotion and vcenter console will tell you the issue. It can be compatibility issue. example: chip set may be different. CD might have mounted. or it could be network.

    storage vMotion - Copying the files from one data store to another data store.

    You can vmotion vm without doing the storage vmotion. You can do the storage vmotion without doing vmotion of vm.

    Vmotion should be within one data center, however you can vmotion from one vcenter to another vcenter.

    Long distance vmotion- You need to have vcenter in linked mode.

    You can join multiple vCenter Server systems using vCenter Linked Mode to allow them to share information. When a server is connected to other vCenter Server systems using Linked Mode, you can connect to that vCenter Server system and view and manage the inventories of the linked vCenter Server systems.

    Linked Mode uses Microsoft Active Directory Application Mode (ADAM) to store and synchronize data across multiple vCenter Server systems. ADAM is installed as part of vCenter Server installation. Each ADAM instance stores data from the vCenter Server systems in the group, including information about roles and licenses. This information is replicated across all of the ADAM instances in the connected group to keep them in sync.

    Max latency max tolerated latency  is 150 ms for Vcenter 6.0
    You need 250 G pipe as well.  

    for VMFS5, One Block consists of 1 MB.  Storage vmotion will impact the performance as it has to copy block by block.

    A VVol datastores has a one-to- one relationship with a storage container and is its logical representation on the ESXi host. A VVol datastore holds VMs and can be browsed like any datastore

    Storage person create LUN and give to VMware admin for configuring Datastore. These data stores are used for creating VMs.


    Snapshot should not use for backup.  It is point in time capture. It is not a copy. If you want a copy then get clone.

    A clone is a new VM that is a copy of the original. A snapshot is a backup of a point in time for that VM, the snapshot allows you to restore to a point in time to the same VM. For example - if you take a snapshot before you change the VM, ie. windows updates, application install, config changes, etc and find that you now are unable to run the VM, you can revert to the snapshot, like it never happened.

    CLONING - Creating a Mirror COPY/CLONE of the virtual machine. SNAPSHOT - Creates a delta file, which enables you to rollback or revert a virtual machine.


    A container object that can contain multiple VM.  

    A vApp is a container, like a resource pool and can contain one or more virtual machines. A vApp also shares some functionality with virtual machines. A vApp can power on and power off, and can also be cloned. In the vSphere Client, a vApp is represented in both the Host and Clusters view and the VM and Template view.

    A vApp allows you to bundle your application with a specific set of definitions. Those could be as complex as multi-segmented virtual networking, or as simple as a start up order. Bundling these as a vApp allows you to take some of the complexity out of the setup process.


    You can create alarm for hosts, virtual machines, etc. etc. Trigger alarm when all conditions satisfied or single one.

    • Vrealize operations Manager
    • vrealize Automation
    • Configuration Manager
    • Hyperic
    • Infrastructure Navigator
    Vrealize operations Manager (old name - VCOPS)
    provides visibility and insights in to performance, capacity and health of environment.

    Health, Risk, efficiency, alerts.

    Badges - Standing on the side and say - look at me.

    it has 4 database. Product/Admin GUI, Collector(REST API), Controller, Analytics, Persistence.

    HA - High availability. The VM will restart there may be brief outage.
    Fault Tolerance - Will take care of node failure.

    VMware Fault Tolerance provides continuous availability for virtual machines by creating and maintaining a Secondary VM that is identical to, and continuously available to replace, the Primary VM in the event of a fail over situation.

    Using Fault Tolerance with DRS: You can use VMware Fault Tolerance with VMware Distributed Resource Scheduler (DRS) when the Enhanced vMotion Compatibility (EVC) feature is enabled. This process allows fault tolerant virtual machines to benefit from better initial placement and also to be included in the cluster's load balancing calculations.

    You can set DRS automation as fully automated.

    Virtual Machine Affinity - keep virtual machines together. Affinity rule is strictly enforced it will allow to set limit for vmotion between hosts . This is effective for license management. Oracle don't recognize this !!@@!

    VMware HA : VMware HA provides high availability for virtual machines by pooling them and the hosts they reside on into a cluster. Hosts in the cluster are monitored and in the event of a failure, the virtual machines on a failed host are restarted on alternate hosts.

    'HA options:
    1) Host based (Default setting)
    2) % based  (most efficient)
    3) Host fail over


    VMware Tools is an optional, free set of drivers and utilities that enhances both the performance of a virtual machine's guest operating system and interaction between the guest and the host.

    DRS should be enabled to ensure that VMotion happens automatically.

    Vmotion has two stages  

    • Pre-copy
    • Checkpoint (Anything changed after the last stage)  

    VMware vSphere Site Recovery Manager (SRM) is a disaster recovery management product from VMware that provides automated fail over and disaster. SRM itself don't do the replication. It requires the VRA or storage based replication.

    SRM is installed as a plug-in to vCenter. Once installed you can see this as a a new management window to your vCenter web interface.

    What is VMware vSphere Data Protection (VDP)? VDP is a robust, simple-to-deploy, disk-based backup and recovery solution. VDP is fully integrated with VMware vCenter Server and the VMware vSphere Web Client.

    SRM required VRA or storage based replication  -  built on Replication ( If I corrumpt my VM then I corrupt the Replication VM as well)

    VDP is backup

    Vshphere Replication (VRA)

    VMware vSphere® Replication is a virtual machine data protection and disaster recovery solution. It is fully integrated with VMware vCenter Server and VMware vSphere Web Client, providing host-based, asynchronous replication of virtual machines.

    VMware Tools is a suite of utilities that enhances the performance of the virtual machine's guest operating system and improves. VMware Tools is installed:

    Shared folders between host and guest file systems,  Scripting that helps automate guest operating system operations etc.

    Update manager - Allow to update the patches for Esxi and vmtools.

    Host profile:

    VMware Host Profiles is a vCenter Server tool for designing and deploying ESX/ESXihosts. Host Profiles works by encapsulating a reference host configuration and turning it into a profile, or template, upon which other hosts or clusters of hosts can be configured

    You create and configure a host then you can use the configuration of this host to configure other hosts.

    Content libraries:

    Are the data store that you can store templates, Vapps etc . Content libraries are container objects for VM templates, vApp templates, and other types of files.vSphere administrators can use the templates in the library to deploy virtual machines and vApps in the vSphere inventory.

    Power CLI

    PowerCLI is a Windows PowerShell interface for managing VMware vSphere. VMware describes PowerCLI as "a powerful command-line tool that lets you automate all aspects of vSphere management, including network, storage, VM, guest OS and more.

    It is same as powershell.

    PowerShell (including Windows PowerShell and PowerShellCore) is a task automation and configuration management framework from Microsoft, consisting of a command-line shell and associated scripting language built on the .NET Framework.

    Basically VMA is the missing service console for ESXi. But it’s more than that too. This allows administrators to run scripts or agents that interact with ESX/ESXi and vCenter Server systems without having to explicitly authenticate each time. vMA can also collect ESX/ESXi and vCenter Server logs and store the information for analysis.


    The vSphere Command-Line Interface (vSphere CLI) command set allows you to run common system administration commands against ESXi systems

    VCLI is same same as power shell.  You need use power shell for image building and auto deploy. vCLI don't have that capability.  

    VCLi give more freedom like linux command. Powershell is same easy to use


    The vSphere Management Assistant (vMA) allows administrators and developers to run scripts and agents to manage ESXi hosts and vCenter Server systems.

    You can download VMA from here -
    vCenter Converter:

    You can convert from Microsoft Hyper-V to eSXi.

    ESXi images:

    software bundle that consists of Core hypervisor, CIM providers, Plugin components and Drivers.  You install this software on each host to allow host connect to Vcenter .

    Auto deploy

    vSphere Auto Deploy can provision hundreds of physical hosts with ESXi software

    You can specify the image to deploy and the hosts to provision with the image. Optionally, you can specify host profiles to apply to the hosts, and a vCenter Server folder or cluster for each host.

    This can be life saver if you have lot of same hardware that required configuration with vcenter .

    Vmware infrastructure bundles: (VIB)

    VIB stands for vSphere Installation Bundle. At a conceptual level a VIB is somewhat similar to a tarball or ZIP archive in that it is a collection of files packaged into a single archive to facilitate distribution. If we look under the covers we will find that aVIB is comprised of three parts: A file archive.

    VIB is created by hardware vendors . It is drivers for installing as part of ESXI images.

    For Auto deploy, instead of iso image you can have PXE-Bootable image. Grab the VIB from hardware software vendors website instead of vmware. Hardware vendors may have the latest VIB,

    You could download HP VIBs from


    Setting reservations is a bad idea !.

    If you set Memory Reservation on a Virtual Machine to avoid memory is over commitment, you must reserve all of the memory allocated to each virtual machine that runs in the VSA cluster.

    vSphere Storage Appliance (VSA)

    It provide HA, DRS, DPM, FT, vMotion etc.

    VMware vSphere Storage Appliance (VSA) is a VMware virtual appliance that packages SUSE Linux Enterprise Server 11 and storage clustering services. A VSA virtual machine runs on several ESXi hosts to abstract the storage resources that are installed on the hosts and to create a vSphere Storage Appliance cluster (VSA cluster). Support around 20 to 35 VMs.

    vSphere Storage Appliance vSphere Storage Appliance (VSA) is an appliance. This aallow access to shared storage at a lower cost.