Test Backup from a Proxmox Environment
TRANSCRIPT
Hello, today we will be taking you through the Nfina-View software’s test backup functionality on a Proxmox environment.
For starters, I’ll show you a quick overview of the setup that we have. We have two Proxmox environments, one being the production on site environment. This has an Nfina-Store VM and five testing VMs currently residing on it. They are currently being replicated via Nfina-Store snapshot replication to the off site location. That is the 9424 RDR site which is also running Proxmox that has no VMs currently running on it except for the Nfina-Store VM.
This is our 9424 production with the five testing VMs on it. If we click into one of these VMs and go to the console, we can see that you can log in like normal just to show that this VM is currently running. Some of these VMS are Linux, some of them are Windows.
Now we’re on the DR environment and you can see that there are currently no VMs residing on this environment, and we’ll be moving those five VMs for testing to that environment.
On the Nfina-View side, if you go to your node Storage and Snapshots tab and to your data set, you can select Test Backup. This first form gives you 2 options to select snapshots, one being the latest number of snapshots that you select or any snapshots older than a specific date. In this case, I’ll select the first option and get the last five snapshots, which you will see that it lists the last five. I’ll be selecting the latest.
This next form shows the snapshot that you selected, the network choices that the share will be mounted to on the prox mock side and also your host options to mount the VMs to, which in our case is just the DR site. You can also change the VM network which I’ll be selecting our backup network for this.
Click next and you can now see on the next form that we have all of our five fail over testing VMs. You have the option to register these VMs or power them all on, which I will do here. You also have the option to disconnect the VM network adapter, skip the VMs entirely, or skip the VM power on. Now you can put in your password to start the test backup functionality.
This next form is just the loading form that can see the tasks as they happen and also watch the loading bar. I will minimize this as the process goes on in the background and you can view the process here from the Proxmox DR environment in the background.
So now we can see that the five testing VMs have been mounted on the DR site and you can see that the clone snapshot name has been appended to the VM to know which clone they are created from.
If we go into the console of the VM, we can see that you can log in as before and that the VM is running just as the one that I showed earlier was. You can also see here on the homepage that all of the original files were as before. Back on the Nfina-View side, you can see that there’s a report that’s been generated for this test backup process. To get to that report, you can exit out of the form, hit settings, and then the Reports tab.
This lists all your reports. Sort by the date generated, find the one that you just ran. In our case it is this one and you can download the report.
Opening the report you can see that it gives us information about the process. It gives you the Nfina-Store node that it was run on, the Proxmox environment that it was run on, the clone, and the data that was generated.
You can see all of our testing VMS are listed here as well with their connection type change to what we selected to the backup network. Also its power status and the IP address of the VM if the agent is installed.
Nfina-View Remote Monitoring Rollback Demo
TRANSCRIPT
For the rollback demo. I have three windows virtual machines, running on a storage volume, which is presented by one of our storage nodes. I have storage volume 01, it’s one terabyte in size and that is running these three virtual machines shown here. What I will do now is run the hive ransomware virus on these machines. If I take a look at my downloads folder, I can see the files that were accessible are now encrypted and have the dot hive extension. Now the ransom note is presented – giving us an onion address with a login and password to pay them the ransom and get the decrypter to the files so we can regain access. So over on the storage RAID side, we were taking snapshots. If we look at the storage volume, we can see we’ve been taking snapshots of the volume every 5 minutes. It’s roughly 2:05 now. So, if I come over here to the Snapshots tab, I can simply select the rollback. I will list the last five snapshots. I’m going back about 10 minutes just to guarantee that we restore before the ransomware is actually executed. I’m selecting this point in time, which will affect the three virtual machines. It will detect that those virtual machines are the one running on that volume. We’re going to choose rollback. We will confirm that we want a rollback and will begin the process where we’re actually going to rollback the entire volume, which will also rollback these three virtual machines over here. At this point, the rollback has been initiated from the storage array side. We’ve started the process of presenting the storage volume back to the ESX host. Once it’s available to the ESX host, we will power on the virtual machines and bring them back online. Now we will re-enable the backup job – and it looks like the virtual machines have been powered back on. What we want to do now is test whether we can actually log in and see if the files are accessible. If I go to my downloads page, I can see that those files are no longer encrypted and I’m able to open the files.
Protect your data with just a few clicks. See the power of Nfina-View for yourself!
Perform a Planned Failover with Nfina-View
TRANSCRIPT
Hi. Thanks for stopping by Nfina Technologies. Today I’m going to give you a quick demo of our planned failover functionality that’s built in to Nfina-View, which is a monitoring management software we have for our storage systems here at Nfina. What you see here is a couple of nodes in this edge cloud hosting cluster. The storage cluster here has these volumes on it that are listed under Zvols.
As far as this failover test is concerned, this particular test volume is what I’m going to focus on. It’s running these five virtual machines on this volume, which you can see here within vCenter on the left. So failover test 1 through 4 and space test and all those virtual machines reside on this failover test data store in Nfina-view.
If I go back to vSphere, you can confirm that failover test 1 through 4 as well as space test reside on that particular data store. The planned failover would be utilized, let’s say, in the event that you proactively want to move your virtual machines from one location to another. The location we’re going to be moving to is actually offsite in a completely different location.
And we’re going to failover to this host here that’s in vCenter, this 233 host. It’s a disaster recovery site. So, when would you use this? Well, it might be of a hurricane is coming or you want to move the VMs to run somewhere else temporarily while you reconfigure the system, something of that nature. To actually perform the failover it’s pretty simple. You’ll go into your Nfina-view interface. If the nodes are in a cluster, then the failover is going to be located at the cluster level, which is represented by this edge cloud hosting with the two-server icon shown here. You go to the failover tab and you find the volume that you want to actually failover.
Now the failover is on the volume level, so any virtual machines that reside on that given volume will actually get moved to the offsite location. To perform the failover, just select the volume, choose failover and that’s going to bring up our failover menu. So there’s a couple of choices here. It’s going to populate automatically with where it’s actually replicating to.
In this case, we’re only replicating to one location. So that’s the only choice. But you do have the ability to replicate to multiple locations. So if you had multiple locations, you would need to select which location you actually wanted to failover to. I have a target choice, which is the iSCSI target that we’re attaching the volume to on the destination side and the source replication.
VIP is what IP address we want to use to replicate back. Once we failover, it will immediately set up replication back to the original system. In this case, I’m going to use the system IP and that will allow it to immediately start replicating back the host options. Here is where you want the VMs to run in the remote location and that will be the 2.233 node over here on the left.
So, with that, I’ll go ahead and click failover. It will bring up this next menu. This is where you have some flexibility to either power on or not power on the VMs, or you can even not register them. It’s going to come by default in the state that they’re currently in. So if they’re currently powered on, power on is going to be checked.
If they’re currently registered, then registrations will be checked here. You also have the ability to skip VM failover altogether that will not do any automation around VMs or you can click the Skip VM power on and that’s just going to clear the check boxes in mass here. I’m going to leave it as default and go ahead and click failover.
So, at that point, it’s starting the failover process. It’s going to shut those virtual machines down. It’s going to un-register them and begin the failover process. You can see in vCenter, it’s un-mounted this data store from both of these hosts on the source side, which means it’s now moved to inaccessible here on the source side. In Nfina-View, let’s check to make sure that any changes have been replicated across that may have occurred during the shutdown. That includes any logging or any data that was written up until the point that the virtual machines were actually shut down. Looks like those changes were replicated across. So, what it’s going to do now is begin to bring the volume up on the destination side.
As soon as the volume is brought up on the destination side, it will immediately go ahead and begin replication back down to the source side even before the virtual machines are brought online here. You can see at this point it registered the VMs and powered them on. You can see our failover test is now mounted here in the destination location and they are now running in the destination location.
Now if I come over to the destination node inside of Nfina View, it failed over to this node. This is the storage node. You can go to failover and see that this DR volume now has these VMs running on here. Now if you want to fail back to the original host whenever you’ve reconfigured or the environmental disasters such as a hurricane or something has passed, you will be able to come back and click failover and that would allow you to fail back to the primary location where it started.
That’s all I have for today. I hope you found this demo useful and you’ll check out more of our demo videos on our site.
Learn how to easily failover from your on-prem location to an off-site location
An untested disaster recovery plan is based largely on assumptions. Backup software may report that a job completed successfully even though the resulting data cannot be restored. A secondary environment may contain replicated servers but lack the networking, credentials, licenses, or application dependencies needed to make those servers operational. Recovery procedures may also depend on employees who are unavailable during an actual emergency.
Disaster recovery testing replaces assumptions with measurable evidence. A properly designed test allows the organization to validate backup integrity, restore applications, exercise failover and failback procedures, measure actual recovery times, identify hidden dependencies, and improve communication among technical and business teams. Nfina emphasizes regular disaster recovery testing as an important part of an effective recovery strategy, noting that organizations should not wait for an actual emergency to discover problems with their backups or recovery procedures. More information is available through Nfina’s disaster recovery solutions.
Testing is particularly important as ransomware attacks, hardware failures, power outages, software corruption, natural disasters, and human errors become increasingly disruptive. An organization may need to recover a single file, restore an entire virtual machine, move operations to a secondary facility, roll back to a clean point before a ransomware infection, or rebuild a complete IT environment. Each recovery scenario creates different technical and operational requirements.
1. Define the Scope and Objectives of the DR Test
The scope might be limited to restoring a single virtual machine from backup, validating a critical database, testing remote access to a recovery environment, or failing an application over to a geographically separate site. A more comprehensive exercise may simulate the loss of an entire data center and require the organization to restore multiple interconnected services in a defined sequence.
The test should also include a specific disaster scenario. Possible scenarios include a ransomware attack, storage-array failure, server-cluster outage, internet disruption, prolonged power failure, accidental deletion, cloud-service outage, or natural disaster. A realistic scenario gives participants the context needed to make decisions and exposes weaknesses that may remain hidden during a generic backup test.
The organization should decide whether the exercise will be announced in advance or conducted with limited notice. Announced tests are useful for training and process development, while surprise exercises can provide a more realistic assessment of readiness. Surprise testing should be carefully controlled to avoid disrupting production systems or creating unnecessary concern among employees and customers.
Success criteria must be documented before the test begins. These criteria may include restoring an application within its Recovery Time Objective, recovering data within its Recovery Point Objective, confirming that all authorized users can access the recovered service, validating data consistency, and completing the test without affecting production operations.
Nfina’s backup and disaster recovery solutions are designed to support recovery planning across virtualized, on-premises, hybrid cloud, and geographically distributed environments. Nfina combines backup infrastructure, immutable snapshots, centralized management, and recovery capabilities that can be incorporated into recurring DR testing programs.
2. Tabletop vs. Technical Recovery Testing
A tabletop exercise is a discussion-based test in which representatives from IT, security, operations, management, communications, legal, compliance, and other business departments work through a simulated disaster. The facilitator introduces a scenario, and participants explain how they would respond at each stage.
The scenario may then reveal that ransomware has encrypted production servers, compromised administrator credentials, and affected the primary backup repository. Participants must determine who has authority to declare a disaster, which systems should be isolated, when law enforcement or cyber insurance representatives should be contacted, and how customers and employees should be informed.
Technical recovery testing validates whether the technology actually works. During a technical test, IT teams may restore files, recover virtual machines, start applications in an isolated network, test database consistency, activate a secondary site, or execute failover and failback procedures. Technical testing frequently identifies configuration problems that cannot be discovered through discussion alone.
A technical test may reveal that a backup is incomplete, a secondary server has insufficient processing capacity, DNS records cannot be changed quickly, application licenses are tied to the failed hardware, or recovery staff cannot access required credentials. These problems might never surface during a tabletop exercise.
Nfina recommends incorporating regular drills into disaster recovery planning and using those exercises to test defined objectives, such as communications, backup systems, and recovery responsibilities. Nfina’s business continuity and disaster recovery resources provide additional information about integrating recovery planning with broader organizational continuity.
3. Validate Backup Integrity
A successful backup notification does not necessarily mean that the protected workload can be restored. A backup job may complete even though application data is inconsistent, files are corrupted, credentials are missing, or the backup excludes a critical disk. The only dependable way to validate a backup is to restore it and confirm that the resulting system operates correctly.
Testing should include file-level restoration, complete virtual-machine restoration, database recovery, application startup, and user-access validation. The restored system should be placed in an isolated environment to avoid network conflicts, duplicate IP addresses, unintended email delivery, or interaction with production databases.
Periodic manual testing adds another layer of assurance. Administrators should select recovery points from different dates, repositories, and storage tiers instead of testing only the newest local backup. Older backups may be required if a ransomware infection or data-integrity problem remained undetected for an extended period.
Version control and retention policies should also be tested. The organization must confirm that it can locate and restore the correct recovery point rather than merely retrieving the most recent copy. Maintaining daily, weekly, and monthly versions improves the likelihood of finding clean data after a delayed incident.
Nfina’s immutable backup technology supports protected recovery points that can be cloned, tested, and rolled back through the Nfina-View orchestration layer. This allows administrators to evaluate backup data without modifying the original protected snapshot.
4. Confirm That Backups Are Protected from Ransomware
The testing team should verify that at least one recovery copy is protected from modification or deletion. This may be accomplished through immutable snapshots, hardened repositories, offline media, isolated accounts, or geographically separated infrastructure.
Immutability prevents protected backup data from being changed or removed during a defined retention period. It can help preserve a known-good recovery point even when an attacker obtains administrative access to production systems. However, immutability should be combined with access controls, network segmentation, multifactor authentication, monitoring, encryption, and off-site replication.
During a ransomware recovery test, the team should practice identifying the last known clean recovery point. Restoring the newest backup without investigation may reintroduce malware or encrypted data. Security personnel may need to review logs, indicators of compromise, backup dates, and application behavior before approving a recovery point.
The restored environment should be scanned and validated before users are allowed to reconnect. Password resets, certificate replacement, security patching, endpoint re-enrollment, and additional monitoring may be required before normal operations resume.
Nfina’s ransomware protection solutions combine rapid failover and rollback capabilities with immutable recovery technology. Nfina-View is designed to help administrators select protected data and initiate failover or rollback without rebuilding the entire environment from the beginning.
Organizations that need an integrated backup platform can also evaluate the Nfina Veeam Backup Server Appliance, which combines backup hardware, Veeam software, immutable storage options, and disaster recovery capabilities.
5. Test Local, Remote, and Cloud Recovery Copies
Local recovery tests should measure storage performance, available capacity, and the time required to restore workloads to production or alternate hardware. The test should account for contention if multiple large systems must be recovered simultaneously.
Remote repository testing should verify connectivity, authentication, encryption, firewall rules, and available bandwidth. The recovery team should calculate whether the network can transfer the required amount of data within the target RTO.
Cloud recovery testing should include more than downloading a backup. The organization may need to provision compute resources, configure virtual networks, apply security rules, restore identity services, change DNS records, and reconnect users. Cloud data retrieval charges and bandwidth limits should also be considered during planning.
Nfina’s cloud disaster recovery solutions support geographically separated backup and recovery strategies across on-premises and cloud infrastructure. Nfina describes cold, warm, and hot recovery approaches, each of which provides a different balance of cost, operational readiness, and recovery speed.
A cold recovery environment generally stores backups or virtual-machine images but requires infrastructure provisioning before workloads can start. A warm recovery environment maintains preconfigured resources and replicated data that can be activated after an incident. A hot recovery environment keeps resources running or synchronized so that services can transition with minimal interruption.
5. Map Application Dependencies
An application may depend on Active Directory, DNS, DHCP, database servers, APIs, shared storage, authentication platforms, certificates, message queues, licensing servers, internet connectivity, and third-party cloud services. If any required component is unavailable, the application may fail even though its primary VM was recovered successfully.
Dependency maps should identify the infrastructure and services required by every critical application. They should also document the correct recovery sequence. Identity, networking, storage, and database systems may need to be restored before application servers, while reporting and development systems may be recovered later.
Business dependencies should be mapped alongside technical dependencies. Restoring an order-processing application may provide little value if employees cannot access customer data, payment services, inventory systems, shipping integrations, or communications tools.
Application maps must be updated whenever software, infrastructure, or business processes change. Cloud migrations, vendor changes, acquisitions, network redesigns, and virtualization projects can all introduce new dependencies.
Nfina’s Professional Services can assist with IT infrastructure planning, virtualization, backup and disaster recovery planning, cloud migration, network optimization, and ongoing management. This type of infrastructure assessment can help organizations identify dependencies and design recovery procedures around actual business requirements.
6. Verify Recovery Infrastructure Capacity
Replicating data to a remote location is not sufficient if the destination cannot support production demand.
During testing, monitor CPU utilization, memory consumption, storage latency, network throughput, and application response times. Compare recovered performance with acceptable service levels rather than assuming that any functioning environment is adequate.
Organizations may intentionally provision reduced capacity in a warm recovery environment to control cost. In that case, the Disaster Recovery Plan should identify which systems receive priority and which services will operate with reduced performance.
The test should also determine whether recovery capacity can be expanded quickly. Cloud and hybrid cloud resources may provide additional flexibility, but provisioning procedures, quotas, images, security policies, and billing permissions must be configured before an incident.
7. Conduct Failover Testing
Before beginning, the team should document the expected sequence, establish a controlled testing window, notify affected stakeholders, and define rollback criteria. The test should avoid creating split-brain conditions in which both production and recovery systems process conflicting transactions.
During failover, the team should record the time required to detect the outage, declare the incident, authorize recovery, activate secondary systems, restore network access, start applications, and validate service availability.
Technical validation should include storage consistency, virtual-machine status, application health, authentication, DNS resolution, firewall behavior, integrations, and user access. Application owners should perform business-level testing rather than relying only on infrastructure monitoring.
The exercise should also evaluate communications. Employees, customers, vendors, executives, and service providers may require different messages during an actual outage. The communications team should confirm that approved templates, distribution lists, alternate channels, and escalation procedures are available.
Nfina-View provides centralized monitoring, management, failover, rollback, and disaster recovery testing for on-premises, hybrid cloud, and multi-cloud environments. Its unified interface is intended to help administrators monitor infrastructure health and manage recovery operations from a single dashboard. More information is available on the Nfina-View platform page.
8. Conduct Failback Testing
Failback is the process of returning workloads from the recovery environment to the repaired or rebuilt primary environment. It is frequently more complex than failover and should never be treated as an afterthought.
Before failback begins, the primary infrastructure must be validated for hardware health, storage integrity, security, connectivity, capacity, and configuration consistency. In a cyber incident, the team must also confirm that the environment is free from the original compromise.
Data created or modified while applications operated in the recovery environment must be synchronized back to the primary site. The organization should define which environment contains the authoritative data and how conflicting changes will be resolved.
Failback testing should document the time required to replicate current data, stop or quiesce applications, redirect network traffic, validate the primary environment, and return users to normal operations. The procedure should include a rollback option in case the primary environment fails during the transition.
9. Test Security Controls in the Recovery Environment
Security requirements remain in effect during a disaster. Recovery environments should not be configured with weaker controls simply because they are used temporarily.
The test should verify identity and access management, multifactor authentication, endpoint protection, encryption, logging, vulnerability management, firewall policies, and privileged-account controls. Temporary recovery accounts should be removed or disabled after the exercise.
Logs from the recovery environment should be available to the security team. This is particularly important during ransomware or data-breach scenarios, when the organization must determine whether the restored systems are clean and whether attackers are attempting to regain access.
Recovery infrastructure should also be patched and monitored regularly. A secondary environment that remains unused for months may contain outdated operating systems, expired certificates, disabled agents, or unsupported software.
The organization should confirm that restored data remains encrypted in transit and at rest and that encryption keys are available through a secure emergency process. Key loss can make an otherwise successful backup permanently unusable.
10. Document Failures and Corrective Actions
Failures discovered during disaster recovery testing should be treated as valuable findings rather than reasons to declare the exercise unsuccessful. The purpose of testing is to expose weaknesses before a real event occurs.
Every issue should be documented while the details are still fresh. Records should identify what failed, when it failed, which systems were affected, how the team responded, and whether the problem influenced RTO, RPO, security, or data integrity.
The team should conduct a root-cause analysis for significant findings. A failed restore might result from corrupted data, missing credentials, expired certificates, incorrect documentation, insufficient capacity, a software defect, or human error. Corrective actions should address the underlying cause rather than merely the immediate symptom.
Each corrective action should have a responsible owner, priority, target completion date, and validation method. High-risk issues that could prevent recovery should be escalated to management and addressed before the next scheduled exercise.
After remediation, the affected procedure should be retested. Closing a task because a configuration was changed does not prove that the recovery process now works.
Test records may also support audits, regulatory requirements, cyber insurance reviews, customer assessments, and internal risk management. Documentation should include objectives, participants, systems tested, results, measured RTO and RPO, failures, corrective actions, and management approval.
Nfina’s Backup and Disaster Recovery Management services can help organizations manage recovery infrastructure, update disaster recovery strategies, and maintain readiness as business and technology requirements change.
Update the Disaster Recovery Plan
The Disaster Recovery Plan should be updated after every exercise. Lessons learned must be incorporated into procedures, diagrams, contact lists, recovery sequences, and system inventories.
Changes should be controlled and approved so that employees do not follow conflicting versions of the plan. Every copy should display a version number, approval date, owner, and next review date.
Updates may include revised RTO and RPO targets, new application dependencies, corrected restoration commands, alternate vendor contacts, additional network diagrams, or changes to staffing responsibilities.
The updated plan should be distributed to the appropriate personnel, and important changes should be reviewed during training. Storing a revised document without communicating the changes leaves the organization vulnerable to the same mistakes during the next incident.
11. Quarterly DR Testing Checklist
1. At the beginning of each quarter, review previous test results and confirm that corrective actions have been completed. Reassess application classifications, business priorities, infrastructure changes, recovery contacts, and regulatory obligations.
2. Conduct a tabletop exercise based on a realistic scenario. Rotate scenarios throughout the year so the organization evaluates ransomware, hardware failure, site loss, cloud outage, network disruption, and human error.
3. Validate a representative sample of backups from local, remote, cloud, and immutable repositories. Restore at least one complete application or virtual machine and confirm that it operates in an isolated environment.
4. Review application dependency maps and verify the recovery order for critical services. Confirm that supporting infrastructure such as identity, DNS, databases, storage, and network connectivity can be restored before dependent applications.
5. Test failover to the designated secondary environment and measure the complete service recovery time. Validate application functionality, user access, security controls, and external integrations.
6. Test failback to the primary environment, including data synchronization, application validation, network redirection, and rollback procedures.
7. Calculate actual RTO and RPO results and compare them with approved objectives. Document any gaps, determine the business impact, and identify required improvements.
8. Verify that immutable or isolated backup copies remain protected from production administrators, ransomware, accidental deletion, and unauthorized changes.
9. Confirm that recovery infrastructure has sufficient compute, memory, storage, bandwidth, and licensing capacity to support the assigned workloads.
10. Review all contact information, escalation procedures, emergency credentials, vendor agreements, and alternate communication methods.
11. Document test results, failures, corrective actions, owners, and deadlines. Update the Disaster Recovery Plan and schedule retesting for any high-risk findings.
Nfina-View can support recurring disaster recovery testing by providing centralized monitoring and integrated failover, rollback, and test functions. Nfina describes the platform as a unified management dashboard for hybrid cloud, multi-cloud, and on-premises infrastructure, helping IT administrators and managed service providers maintain visibility across protected environments.
Building a Tested Disaster Recovery Strategy with Nfina
Nfina Technologies provides server, storage, hyperconverged, hybrid cloud, backup, and disaster recovery solutions designed to support business continuity from the edge to the cloud. Its portfolio includes immutable snapshot technology, Nfina-View centralized management, Veeam-based backup appliances, professional services, and cloud disaster recovery resources.
Nfina’s disaster recovery architecture can support failover, rollback, backup testing, and geographically redundant data protection. Nfina-View provides tools for centralized infrastructure monitoring and simplified DR testing, while immutable snapshots help maintain recovery points that cannot be modified during their protected retention periods.
The most effective disaster recovery tests are realistic, repeatable, measurable, and documented. They validate both technology and people, measure the complete recovery process, and turn every failure into a corrective action.
Organizations should never wait for a ransomware attack, hardware failure, or natural disaster to discover that a recovery process does not work. By conducting regular tabletop exercises, restoring protected data, testing failover and failback, measuring actual RTO and RPO, and correcting identified weaknesses, a business can enter a real emergency with evidence that its recovery strategy is ready.
To learn more about designing and testing a resilient recovery environment, visit Nfina Technologies, explore Nfina’s backup and disaster recovery solutions, or review the company’s cloud disaster recovery services.