Data Engineering Services

Description

Galtronics wanted to support the Green IT technologies by providing their users with a tool that could help to analyze power consumption without digging through paper bills. The company turned to Altoros to build an energy management system for collecting, storing, and processing petabytes of power consumption data. The solution also had to easily scale up from a single server to thousands of machines without compromising the overall performance.

The outcome

Altoros suggested using the Cassandra NoSQL database and the Hadoop distributed computing system for building a highly scalable system storing and processing vast amounts of data. Cassandra can write to a data store 2,500 times faster that MySQL solutions. Cassandra enables power consumption data to be gathered and aggregated in a fraction of a second. Hadoop was implemented to cope with the big data management challenge. It uses clusters of computers and distributed processing to analyze petabytes of power consumption records. The solution makes it possible to collect, aggregate, and process power consumption data in real time. symHome also generates detailed reports on how electricity is being consumed. The Web site can be accessed from any PC or Apple device. Users can quickly identify which appliances and devices cost them most money. The system can be scaled next to endlessly as the number of connected houses grows. The solution helps the customer achieve its Green IT goals by continuously reducing the environmental impact of their products and operations.

Description

The customer is a start-up company that came to Altoros with an idea to develop a service similar to “virtual FedEx” that delivers electronic documents. The main advantage of the emerging system was to be a simple and intuitive user interface. Ease of navigation and availability of the planned features were tested on a prototype developed by Altoros.

The outcome

It was decided to use cloud technologies to ensure high availability, quick response times, and easy horizontal scalability. The system was integrated with EMC, FileTransfer services, and a third-party Reporting Tool selected by Altoros’s experts.

The customer released a new solution that makes document exchange easier and more efficient. The system has already become in demand, thanks to a user-friendly interface, smart navigation, enhanced security, and unlimited storage space in the cloud. By testing prototypes, we were able to carefully study all requirements, confirm functionality, and access usability before we started development.

Description

NuoDB is a NewSQL database platform with a unique geo-distributed database management system. It provides linear scalability, outstanding scale-out performance, continuous availability and can run anywhere—in a local data center, in multiple data centers around the world, or in the cloud. With support for multiple platforms, users are able to build hybrid solutions that have both Linux and Windows hosts.

The outcome

The NuoDB Migrator enables users to extract data from SQL-compliant databases and load it to NuoDB. The tool is able to manage large data processing scenarios, such as failures, continuation, restart and catch-up. During operation, completion status is displayed for task execution and users are provided detailed reports on the migration process.

While many enterprises would prefer to move to scalable NewSQL databases like NuoDB, the difficulty of data migration has held back progress. But with NuoDB Migrator and NuoDB Applier for Tungsten Replicator—developed by Altoros—it is now easy to jump on board with NuoDB’s geo-distributed database. The NuoDB Migrator is cross-platform and ensures a high level of data consistency and integrity, eliminating the old complications of data migration.

Description

The cloud infrastructure provider turned to Altoros to do independent performance tests on their virtual machines and provide recommendations on how to make the system more efficient. The results of our assessment revealed that the system’s performance was in fact 20-30% higher than the results provided by the customer. Our engineers also drew up a list of recommendations on how to improve the system’s efficiency and gain competitive advantage.

The outcome

According to Altoros’s tests, a virtual machine with Ubuntu Linux installed processed 1 TB of TeraSort test data in 13.65 minutes, which is 1.2 times faster than in the customer’s test results. Featuring enhanced CPU bursting and improved disk input/output speed, virtual machines with custom OS installed were able to complete the same task in 6 minutes, which is 2.75 times faster than the results demonstrated during the initial benchmarking. The tests revealed that non-optimized Linux machines become unstable, if a cluster exceeds a certain size. The reports, instructions, and scripts provided by Altoros can be later used by the customer’s team to replicate the test results or to improve the system’s stability.

Description

LikeFolio shows users information on the top publicly traded companies most talked about on their networks. It also predicts how a portfolio of their securities worth $10,000 would perform over a period of 12 months. SwanPowers, LLC turned to Altoros to apply best practices in cloud and Ruby development to their existing system in order to satisfy the high requirements to the level of concurrency, load, response times, etc.

The outcome

Having analyzed the infrastructure, software architecture, and application code, we were able to implement improvements that have resolved all the issues mentioned above. The optimization included reconfiguring the Amazon EC2 instances, implementing a Redis pub/sub mechanism to decouple jobs from DB operations, managing social networks quotas internally, so that the application would never exceed the allowed number of requests, and much more. Thanks to the improvements introduced by our team, the customer was able to launch several marketing campaigns without being afraid of performance issues due to increased traffic. The application’s overall uptime and performance have been improved significantly. Taking into account the LikeFolio’s focus on social networks, Altoros made sure that it complies with the required API quota/limits.

Description

The system of a European Wi-Fi Internet provider was significantly updated. The team added a Google-like search, refactored the architecture, and updated queries to the database to improve performance. In addition, the new solution serves as an intelligent tool that analyzes user preferences and helps to improve promotional strategies and create targeted advertising and content. It can be used to evaluate quality of services, gather statistics, and build detailed reports.

The outcome

The system performance increased by 1,000 times due to updated architecture, optimized queries, and improved search. The customer was impressed by the suggested solution and the improvements they were able to achieve. In order to test system performance and feasibility of the project, Altoros decided to develop a prototype. The team used three servers provided by the customer in a master-slave arrangement with the Puppet management solution for configuration and management. We uploaded all the content to a different storage to separate user authorization functionality from search queries. Solr was used for advanced search through the content. As the result, users can now search content by text as well as by tags. The system features auto suggestion and various data filters. New information is regularly uploaded by the crawler that visits a list of specified Web sites. Then it indexes and adds all this data to the content database. Users can search content in three languages, including English, French, and Dutch.

Description

A Web-based system that enables retailers to gather statistics on consumer demand for their products and adjust their sales strategy according to these figures. TapMap is a start-up company that needed a solution to synchronize retailers’ Point of Sale systems with online accounts. The solution had to include a Web system providing information on stock and prices. The information was to be updated on a daily basis and accessible to customers through free iPhone and Android apps. The customers would scan barcodes with their smartphone cameras to compare prices while the Web system gathers statistics and generates reports for retailers. Apart from that, searches and scans were to be displayed on a visual map in real time.

The outcome

Our team of Ruby on Rails developers analyzed the architecture of the solution and found a way to improve it. The connections between the objects were optimized to enhance the structure of the database. This greatly improved the overall performance and provided nearly endless scalability.

The customer was one of the three winners at the International EXPO 2011. The iEXPO featured a number of promising startups who presented their projects to the audience.

Description

The customer turned to Altoros to improve its biotechnology system that analyzes DNA samples for mutations on early stages. The legacy tool was able to de-duplicate only 1,000 samples maximum—due to memory and CPU limitations—and it still took hours (or even days) to process the pipeline. The goal was to fix performance bottlenecks as well as enable linear scalability for processing 10,000+ bio samples at a time.

The outcome

Altoros has delivered a highly scalable analytical system for de-duplication of genome samples—as a part of the customer’s analytical platform. Thousands of hospitals and laboratories worldwide use the system to detect DNA mutations, saving thousands of lives. The analysis takes minutes now, not hours; it allows for processing 10x more genome samples—compared to performance of the legacy system. Altoros’s engineers have also proposed a reference architecture for updating a reporting solution. Inspired by our recommendations, the customer went on improving the system with open-source data analytics technologies, which will eventually allow for saving thousands of dollars.