Wayfast Deployments Deep dive

INTRODUCTION SETTING ENV FOR EXECUTING A DEPLOYMENT

Wayfast applications are hosted on instances of SQL Server and IIS. Larger customers are housed in dedicated instances of SQL Server and IIS, while the remainder reside on shared instances of both. Wayfast maintains the segregation of customer data on shared instances via customer-specific SQL logins, databases and schemas. Customer website assets reside in customer-specific IIS websites and virtual directories tied to a customer-specific app pool and service account.

Open

Figure 1 Anatomy of a Wayfast Application

Wayfast applications consist of:

·         Metadata and objects (such as stored procedures, views and triggers) stored in SQL Server

·         Executables such as C# assemblies and PowerShell scripts.

·         A skin consisting of CSS, JavaScript and HTML files.

The Wayfast rendering engine translates the metadata into the web-based application our customers experience. The deployment process (outlined below) automates the configuration of IIS folders and .NET applications, and populates the customer’s SQL Server database. Consequently, Wayfast development and deployment involves creating and populating databases, upgrading them as development proceeds, and verifying that Wayfast has deployed customer’s application correctly. 

WHO PERFORMS THE DEPLOYMENT?

Most projects are synchronized by developers with the permission of the project owner. Production customer deployments have the added requirement of coordination with the customer prior to deployment in order to mitigate the chance of breakage.

INITIATING PROJECT DEPLOYMENT

After activating your task in Wayfast and selecting a project, click the “Play” button on the menu bar. Then proceed to designate an action type and target (i.e., the customer schema).

Open

Figure 2 Project Deployment Synchronization Page

SELECTING ACTION TYPE AND TARGET CUSTOMER

The table enumerates the deployment action types displayed in the “Type of Action” dropdown.

Table 1 Deployment Action Types

After designating an action type, select a target. The target identifies the customer database schema (e.g., dbo). The schema functions as a namespace within the database, ensuring the separation and access security of customer data.

Figure 3 Anatomy of a Wayfast Deployment

Deployments are kicked off on a specific schedule or via a Wayfast workflow.

Figure 4 Scheduling a Deployment

Now we need to select an target environment for this deployment. In wayfast we call that customer.

It could be an internal environment like QA or it could be a client-specific environment like SomeCompanyProductionEnv.

We can reference development environment using “DBO” customer.

Then we need to add a comment for future reference and tacking. Let's say: Changes for user story 1234.

When I select the customer can I click “Objects shared with other projects” to understand the impact of the changes I'm deploying.

This is just a warning for developers to make sure they are aware that the application shares objects with another app so they can check it and update it if needed.

If the app I'm deploying relies on a Database that's shared with another project we need to make sure that changes in my current project are not unexpectedly breaking the other project.

For now let’s assume that this is a safe change.

Last, we click the “start” button to trigger the synchronization process.

POSSIBLES ISSUES DURING SYNCHRONIZATION: ACTIVE TICKETS

Deployments do not complete for a host of reasons, most often because of active development tickets. For the deployment to proceed, all tickets must be closed or marked as pending.

Wayfast neither performs static analysis nor versioning but does capture a history of each change made. The platform likewise does not facilitate isolation of changes via code branching. It falls to developers to ensure stored procedure changes do not break downstream environments. The ramifications of the all-or-nothing nature of Wayfast development (on what constitutes a single code line) means that unless measures are taken outside of Wayfast to pause development while a feature is in QA, there exists no way to fix bugs in the same code version, retest that version and deploy that version.

HOW TO ROLL BACK IF SYNCHRONIZATION BREAKS

The steps taken to rectify a synchronization error depend on the nature and severity of what broke. Wayfast employs log shipping for database backups. Frequency varies from customer to customer. QA, being less reliant on specific data being present in the database, uses this approach to restore prior database versions. When data loss is acceptable a DB backup can be restored to go back to a previous version in a simple and easy way.

Rolling back production databases can pose a greater challenge when customer data is involved. Simple schema changes that do not affect data are readily rolled back by simply applying scripts that take objects to a previous version, often with brief delays while SQL Server temporarily locks the affected SQL objects. Examples of this kind of changes are adding fields to a table, renaming them, removing fields with no data or changing a Stored Procedure.

When a simple rollback entails the possibility of the loss of customer data, it is incumbent on the developer to develop a manual SQL script to capture any data changes made after the most recent database backup. Briefly, the order of operations requires the developer capture the current (modified) state of the customer data, restore the database from a backup, determine the data changes between the backup and the current data, and apply the script to bring the data up to date.

When determining the delta between the current data and the backup, the developer needs to take account of the four table types in Wayfast.

Table 2 Wayfast Table Types

Partitioned Data Considerations

While generally not part of database synchronization and rollback, partitioned tables – and specifically transaction tables – may need attention after a rollback. Wayfast performs periodic partitioning of rapidly growing tables by copying historical data into a growing array of tables joined together with a view. This manual portioning (contrasted with SQL Server’s automated approach to partitioning) happens periodically, controlled by Wayfast. Even when a partition occurs after backup, the restored view should retrieve the same data as before. Any new tables and resituated data resulting from the partition will be absent from the view. The data itself will be in the original transaction tables and these may need attention to avoid the loss of recent data.

SYNCREPORT AND SYNCHRONIZACTION DETAILS

The Wayfast platform Syncreport details the performance and outcome of synchronizations performed across the Wayfast platform. You can access this report via the platform menu in the upper righthand corner of the Wayfast screen.

Along with high level information such as synchronization type (see “Action Type,” above), customer, project and process outcome, the report provides links to the individual SQL scripts and log entries, along with listings of the controllers and databases affected by the synchronization.

The report supports filtering by action type and customer, as well as a quick text search. Along with the customary log entries (e.g., status, outcome, start/stop times, &c.), the listing includes a detailed account of every object created or modified.

Wayfast synchronization leverages the Redgate SQL Provision deployment tool. The tool compares the source and target databases and computes the requisite modifications needed to bring the target into conformity with the source. Certain conditions cause synchronization to stop. Examples include modifying a VARCHAR to an INT column. If the table is empty, the modification will proceed. Otherwise, the synchronization stops with an error, leaving to the developer the responsibility to save off the data, empty the table, run the synchronization and then reinsert the saved data taking care to convert the VARCHAR column data to INT.

Syncreport also provides details on unexpected exceptions thrown during synchronization. Common exceptions include primary key violations, foreign key constraint violations, timeouts, and data comparison failures.

Finally, Syncreport provides links to download the SQL scripts that ran during synchronization.

ACHIEVING MULTITENANCY THROUGH SCHEMA NAMING

Figure 7 Project General Options

The Customers page under the General Options icon displays the customer associations for the current project. Click on the “New” button in the upper righthand corner of the customers grid to call up a list of customers. For example, to add CustomerX as a target for a project, filter the list on CustomerX, select one or more entries, then click the submit button. Returning to the project deployment synchronization page, you will find CustomerX included in the list of available synchronization targets (see Figure 2 Project Deployment Synchronization Page).

Figure 8 Associating a Project to a Customer

During development, developers use the SQL Server dbo schema. For example, if the project requires an “Employee” table, the table should be named “dbo.Employee” and all subsequent references to the table (in views and stored procedures, etc.) should include the dbo schema. During database synchronization, Wayfast replaces the dbo schema with the target customer schema. Thus, references to dbo.Employee would be rewritten as CustomerX.Employee.

In future, Wayfast will support project branching and versioning whereas today it does not.

WAYFAST SECURITY MODEL

Wayfast uses customer-specific database user accounts and schemas to achieve data segregation and security. Thus, CustomerY will have its own user and schema, CustomerX will have its own user and schema, CustomerY will have its own user and schema, and so on. Each user has access only to that customer’s schema and each schema is designated as the default in the customer’s Wayfast database. Customers cannot access the dbo schema.

Backend components that supply services to Wayfast – for example, file and table uploaders – may require schema-agnostic database access. These components query a configuration database for connection strings that, in turn, provide a way for the components to access customer schemas.