B A C H E L O R P R O S J E K T - student.cs.hioa.nostudent.cs.hioa.no › bachelorprosjekter › data › 2017 › 39 › docs › Fina… · II.1.1.2 Kanban 10 II.1.1.3 Scrumban

Studieprogram: Informasjonsteknologi Postadresse: Postboks 4 St. Olavs plass, 0130 Oslo Besøksadresse: Holbergs plass, Oslo

PROSJEKT NR.

TILGJENGELIGHET Offentlig

Telefon: 22 45 32 00

Telefaks: 22 45 32 05

BACHELORPROSJEKT HOVEDPROSJEKTETS TITTEL Infront Business Intelligence

DATO 24.05.2017

ANTALL SIDER / BILAG

PROSJEKTDELTAKERE Eirik Lunnan Djuve Simen Dagfinrud

INTERN VEILEDER

Christer Hadland

OPPDRAGSGIVER Infront AS

KONTAKTPERSON Espen Øverbye

SAMMENDRAG

Application developed for collecting, processing and presenting business intelligence. Intended for the client and their providers administrators.

3 STIKKORD Angular 2

Business Intelligence

Financial Technology

Infront BI Final report

Preface As a part of our bachelor project, we are required to provide documentation of the work conducted. It's purpose is to give insight in the result product, our development process and our decisions regarding technology. It includes an introduction, process documentation, product documentation, result and appendices. For readers not familiar with financial technology, it is recommended to read the appendices before the actual report.

1


Table of contents

Preface 1

I Introduction 5 I.1 Client presentation - Infront AS 6

I.1.1 Status 6 I.1.2 Contacts 6

I.2 Group presentation 7 I.3 Project background 7

II Process documentation 8 II.1 Project planning 9

II.1.1 Choosing the right method 9 II.1.1.1 Scrum 9 II.1.1.2 Kanban 10 II.1.1.3 Scrumban 10 II.1.1.4 The right method 11

II.1.2 Timeline 11 II.1.3 Project tools 12

II.1.3.1 Office Suite 12 Microsoft Office and Microsoft Office 365 12 Overleaf 12 Google Drive 13

II.1.3.2 Organization 13 Trello 13 Microsoft Project Professional 2016 14

II.1.3.3 Source Control 14 Git 14 Git CLI 14 Atlassian SourceTree 15 Atlassian BitBucket 15

II.1.3.4 Communication 15 Skype For Business 15 Remote Desktop 15

II.1.3.5 Integrated Development Environments 15 Microsoft Visual Studio 15 JetBrains WebStorm 15

II.1.3.6 Testing tools 16 Postman 16

II.2 Cooperation agreement 16

2


II.3 Development 16 II.3.1 Research Phase 16 II.3.2 Planning at Infront 16 II.3.3 Back-end 18 II.3.4 Front-end 19

III Product Documentation 21 III.1 Requirements 22

III.1.1 Initial requirements 22 III.1.1.1 Must 22 III.1.1.2 Should 22 III.1.1.3 Could 22

III.1.2 Additional requirements 22 III.2 Dependencies 23

III.2.1 ASP.NET Core 23 III.2.2 Infront web-toolkit 23 III.2.3 Angular 2 25 III.2.4 Onsen-UI 26

III.3 Product description 27 III.3.1 Data 27 III.3.2 Logging 28 III.3.3 Back-end 29 III.3.4 Front-end 31

Desktop application 32 Mobile application 34

III.4 Implementation 35 III.4.1 Logging 35 III.4.2 Log Service 36 III.4.3 Front-end 36

Angular Modules 36 Components 37 Services 39 Mobile templates 39 Webpack 40

III.4.4 Design 41 III.4.5 Database design 41

III.5 Result 43 III.5.1 Client feedback 43 III.5.2 Discussion 43

IV Appendices 45

3


IV.1 Financial technology 46 Markets 46 Stock trading 46 Indices 46 Stockbrokers 47 Ticker symbol 47 Market data representation 47 Market data distribution 48

IV.2 Software requirement specification 50 Introduction 50

Scope 50 Definitions 50

Overall description 50 Product perspective 51 User characteristics 52 Constraints 52 Assumptions and dependencies 52 Organization of requirements 52

Specific requirements 53 Functional requirements 53 Non-functional requirements 56

Must 56 Should 57 The front-end web application should be a single page application, built using Angular 2, and bundled using Webpack. 57

IV.3 Gantt Diagram 58 References 59

4


I Introduction

Bachelor Project 2017

Oslo and Akershus University College of Applied Sciences Group 39

5


I.1 Client presentation - Infront AS Infront AS is a software company specialising in financial information and trading, based in countries all over the world, with a customer base composed of European financial professionals. Their main product is the Infront terminal, giving traders, investors and news-agencies access to global real-time market data, trading, news, and analytics of key markets. They also develop a web-application, from here on referred to as the WebTrader or WT, which will serve as the main entry point for the Infront BI project.

I.1.1 Status Infront's goal is to by 2020 be within the top three market data providers in their segment, and is working hard to enhance and extend their product line-up. One of the most recent addition is offering treasury data. They have also extended their target markets to include South Africa, opening new offices in Cape Town and Johannesburg. Infront's main software product is a Windows based market data terminal, used by finance professionals across Europe. Infronts products are mainly sold business-to-business or through relevant providers, i.e. stockbrokers and banks.

I.1.2 Contacts Two contacts were provided by Infront:

Name Role Email

Ole-Martin Grøtterud Client supervisor [email protected]

Espen Øverbye Client contact/stakeholder [email protected]

6


I.2 Group presentation Eirik Lunnan Djuve is the project’s student lead, and a Applied Computer Technology bachelor student at the Oslo and Akershus University College of Applied Sciences. He has worked part time as an apprentice front-end developer for the client, Infront, since February 2016, and will be writing the front-end aspects of the project. Simen Dagfinrud is a Software Engineering student at the Oslo and Akershus University College of Applied Sciences. He will be writing the back-end aspects of the project.

I.3 Project background The idea of making a Business Intelligence prototype as a bachelor project was started shortly after Eirik, started working part time at Infront in February 2016. Infront has previously facilitated projects for master’s degree level theses in finance, and internships in both finance and technology, but never a bachelor project. Partly due to the domain knowledge needed to work with financial technology. The project was drafted based on discussions between Eirik and stakeholders at Infront. The final draft resulted in a description outlining a project further developing customer usage analysis. Multiple possible tasks were discussed, taking into account previous experiences and the status previously described. This task was selected mainly due to the following five acknowledgements:

● Finding an existing solution for providing business intelligence to Infront's providers has proved difficult.

● Recently, Infront expanded their product line-up to include a web application, offering a scaled down version of their main product for non-professional users. As this is a new user segment, the need to understand how customers use their products has been amplified. As a part of developing this application, Infront have developed their own web-toolkit. The toolkit is currently only used by the WebTrader, but the intention has been to utilise its capabilities in other products as well.

● Single page web-applications has many advantages to multi page. Creating a single page web-application, using Infronts web-toolkit, is desirable to identify any challenges in converting existing applications to become single page.

● Scope could be adjusted based on progress.

● The assignment should be possible to conduct with a high level of independence. However, succeeding will warrant cooperation with key resources at Infront.

7


II Process documentation



8


II.1 Project planning

II.1.1 Choosing the right method The requirements for the solution is split into "must", "should" and "could" requirements, with the expectation that the group would not be able to complete all tasks. As the possibility to adjust scope based on progress was important to this project, the project management framework selected would have to permit adjusting plans as the application is developed. Traditional waterfall type methods, where most of the planning is done in advance would therefore not be applicable. To be able to plan a project without a predefined end goal, we would have to use an "Agile" project management method. Agile is defined as "The ability to create and respond to change in order to succeed in an uncertain and turbulent environment" (Agile Alliance, n.d.). Agile software development methods follows the "Agile Manifesto", a set of values agreed upon by the creators of the term "Agile" in 2001 (Agile Alliance, n.d.)

● Individuals and interactions over processes and tools ● Working software over comprehensive documentation ● Customer collaboration over contract negotiation ● Responding to change over following a plan

We have researched the most used agile development frameworks Scrum and Kanban, along with the hybrid Scrumban, and considered their differences and applicability for this project.

II.1.1.1 Scrum Scrum is the leading agile development methodology, and an obvious candidate. Tasks are organized in a product backlog, representing the work needed to complete the product. Initially this list contains initially known and best-understood requirements, and new requirements are added as they are identified. Development is conducted in iterations, called sprints, with a fixed time-span lasting no longer than 4 weeks. At the start of each sprint, a suitable amount of tasks are selected from the project backlog, and transferred to the sprint backlog. These tasks are to be completed within the timeframe of the sprint. Each sprint is concluded by a sprint review and retrospective. Each sprint should produce a potentially shippable result. Meaning it should either be ready to be put into production, or at a state demoable to stakeholders to receive feedback (Scrum Alliance, 2014). The Scrum Framework defines three specific roles

● Scrum Master, acting as facilitator keeping the team focused on its goal

9


● Scrum Product Owner, prioritizes the backlog and handles all communication with other stakeholders.

● Scrum Team, a small team of people working together. The ideal size is 5-9 persons. Our main issue with Scrum is the idea of time-boxing all tasks. As novice developers, estimating time is difficult without due to lack of experience. The amount of dependencies involved would make it even harder. Scrum also involves a fair amount of formal processes.

II.1.1.2 Kanban Kanban is among the other leading agile methods. Both the name, and the idea, originates from Toyota's Production System established in the 1940s. Kanban is japanese for "card" or "visual signal", and refers to the cards Toyota line-workers used to signal steps in their manufacturing process. Kanban emerged as an Agile management process for knowledge work in 2005.(Leankit, n.d.) Kanban focuses on creating a continuous workflow, relying on the principle "just in time". Work is to be executed only when demanded. The workflow all team members are to be visualized in a visual model, creating a shared overview of the process. The idea being that it will be easier to identify workflow problems and bottlenecks, and easier to keep track of what other team members are doing.(Kanbantool, n.d.) The visual model of a Kanban project is typically organized on a "Kanban project board". A kanban project board consists of three or more columns, at least including "Backlog", "In Progress" and "Done". Other column can be added as needed, relative to the workflow of the process. Typical columns could be "Impeded", "Ready for testing" and "Verification". Another important principle is to limit work in progress. This is to limit waste, originating from multitasking and switching between tasks. Issues related to multitasking is plainly apparent when visualized on a Scrum board, as to many cards in the "in progress" column would clutter the board. This sort of lightweight, progress oriented agile method we believe is appropriate for this project. It's only major flaw being the total emission of the concept of time. As this is a bachelor project, with a fixed due date, some sort of time-boxing will have to be included in our project planning.

II.1.1.3 Scrumban Scrumban was introduced to the team by the client, as it is their preferred agile project management methodology. Scrumban is a hybrid, based on Scrum and Kanban, that emerged as teams were using Scrum for developing projects, while at the same time using Kanban for maintenance projects. (Leankit, n.d.) Scrumban uses the workflow of Kanban, where tasks are continuously started when there are resources available in the team. This fits the goal to adjust scope based on progress, rather than time-boxing them in advance.

10


Iteration planning is conducted like in Scrum. Focusing on the prioritization of tasks. Task are prioritized not just by what tasks are the most important, but on what it is best to work on next. Tasks has to be prioritized to ensure the requirements deemed most important are completed, while at the same time taking into the account the progress of other dependencies. To make this clear on the project board, it includes the "ready" column between the backlog and doing columns. This column is commonly labeled "Selected for execution" at Infront. Scrumban also recognizes the use of time-boxed iterations when suitable. Although we don't want to time-box every task, milestones can be used to evaluate the progress of the project. If a task is taking too long, and doesn't affect any of the "must" demands, it should be moved down the prioritization list. The goal of the project is a proof of concept, meaning the experience gained from trying can be as valuable as completing it.

II.1.1.4 The right method We have chosen not fully commit to any management method, as we believe it would be counterproductive for a project of this size. The team only includes two developers, meaning resource coordination should be manageable. We will conduct our work at the client facilities, and have the opportunity to communicate with stakeholders at a daily basis. We believe this greatly lowers the need for formal processes meant to ensure the product being developed matches the client's expectations. We have chosen to use principles and methods from Scrumban deemed suitable for the project, resulting in a highly agile development process aiming to spend as little time on formal processes as possible. This also fits the client’s general practices and conventions of spending as little time on formal tasks as possible. We have agreed to follow the following guidelines:

● Workflow will be visualized, using a digital kanban board. ● Tasks will be added to the backlog continuously, when identified. ● The backlog will be sorted based on priority. ● Retrospects will be held at the planned milestones to evaluate progress. The backlog will

be reprioritized as needed. The Client Supervisor will participate at these meetings as Product Owner.

II.1.2 Timeline

Figure 1: Timeline of original plan. See full version in Appendix 3.

11


Milestones were planned for each major task, as well as due dates. A generous five week period was reserved for end documentation, integration and deployment at the end of the development phase. It was also meant to be a buffer, to have the time to finish any ongoing or uncompleted tasks. No new tasks were to be started at this time. Two weeks were reserved for planning and design before starting development. In this period we planned to fully identify the main tasks needed to fill the clients requirements, research dependencies and make technology choices. The development phase includes three milestones, marking the end-dates for the development phases for the major back-end components. These are not meant as rigid "feature complete" dates, but rather planned retrospectives for reevaluating how to proceed.

II.1.3 Project tools

II.1.3.1 Office Suite

Microsoft Office and Microsoft Office 365

For the purposes of writing reports for the bachelor project, some form of online, synchronized office suite with the ability of multiple users editing the same file simultaneously was deemed necessary. Initially, the group attempted to use Microsoft Office and its associated web-suite Microsoft Office 365. This attempt was prone to instability on the desktop platform, issues with performance in the browser, and required external or additional storage for synchronization, making the task tedious at best and perilous at worst with consideration to preserving work. It was decided to look for other alternatives as the problems with these solutions mounted quickly in the early stages of writing the pre-project report.

Overleaf

Overleaf, a web-application for editing the popular typesetting language LaTeX, was briefly considered as a tool for writing the project reports after the Microsoft suites were shown to be ineffective. Overleaf would have provided excellent control of the final documents' appearance through programmatic typesetting in LaTeX, and allowed for source control to be used in the same way it was used for code in the project. It was quickly determined that LaTeX in general would require too much time investment due to its steep learning curve and general complexity. Overleaf also lacked notifications for users when their files were altered or commented on. Another solution was sought.

12


Google Drive

Google Drive was suggested, as there had been previous experience in the group using it without complications. Its “Word-like” editor, Google Docs, added additional functionality such as the ability to switch to a "suggestion" mode, allowing the group to comment on and accept or deny suggestions made. Deletions, edits, and additions could be coordinated well and the original document preserved until a decision was made using this feature, making seamless cooperation simple. Google Docs also provided the group with easy access control, so as to allow the client’s and university college’s supervisors to view and comment on the documents without disrupting workflow. Google Docs’ “assign to” feature also served as a useful way of distributing work within a single document, and a reminder that there are action items to attend to within the document, making prioritization easier. These traits, coupled with free storage for the documents, simple synchronization that worked well as multiple users browsed and edited the same document, resulted in the group choosing Google Drive as its primary office suite for the project from then on. Google Drive, in addition to its stable and well tested web-application, was integrated well with their e-mail system, desktop application, and mobile apps, greatly simplifying cooperation on textual documents.

II.1.3.2 Organization

Trello

Trello shows lists of cards that can be assigned to users, given tags, and can be commented on. It is a highly useful organizational tool that was invaluable in planning and prioritizing work as the project progressed, giving group members, client supervisor, and the university college supervisor insight in how the group was working and on what.

13


Figure 2: Trello image depicting the last stages of the project The group used Trello to organize workflow by dividing it into several lists; a backlog list, a staging list, an in-progress list, a testing list, and a list of completed cards, representing one or more features or fixes. Infront already used Trello to organize its workflow, and allowed key developers to communicate with the group on integration and problematic situations involving features. In addition, it was prominent in status meetings, giving an overview of the group’s progress and allowing in-meeting planning for future “cards” added.

Microsoft Project Professional 2016

Allows for creating complex diagrams for project milestones and completion, used for documentation and organizational purposes.

II.1.3.3 Source Control

Git

The primary, encompassing technology for the group’s source control is Git; a distributed version control system that is flexible enough to be used in both large and small projects, and highly adaptable to one’s needs and requirements. This was also the encompassing technology used by Infront, and it is the current gold standard for source control.

Git CLI

Git CLI was predominantly used on the back-end side of the project, due to the comparatively few files relative to the front-end, and the verbosity of error messages produced should there be any. To some degree due to the back-end already needing to use CLIs with database testing and starting .NET Core based applications.

14


Atlassian SourceTree

Atlassian SourceTree is what most developers the group interacted with used as an interface with Git, and was predominantly used by the front-end, where the sheer amount of files made it difficult to sensibly use the command line.

Atlassian BitBucket

The group was given access to source code and a repository at Atlassian’s BitBucket online repository, allowing for two-factor authentication and sharing among the group members as well as developers relevant to the project.

II.1.3.4 Communication

Skype For Business

Skype for Business allowed communication remotely while coding, and was one of the primary means of communication with the Infront developers for questions, and finding potential bugs. Its functionality was highly useful for working remotely, as it let the group share their screens and demonstrate exactly what they meant while looking at a code base or an issue in organization/project work.

Remote Desktop

Microsoft Remote Desktop was utilized in situations where it was necessary to enter Infront’s internal network for testing from home, or accessing the staging server used for testing the complete system. It allows remote access from one Windows computer to another, through directory server access and configuration on the host and client computers.

II.1.3.5 Integrated Development Environments

Microsoft Visual Studio

Visual Studio served as the primary development platform throughout the project. This was primarily due to Infront utilizing Microsoft technologies, and to ease integration as well as make full use of the developers’ help and knowledge throughout the project, as this was the IDE they were the most familiar with. Both the back-end and the front-end utilized Visual Studio heavily when dealing with ASP.NET Core and .NET Core.

JetBrains WebStorm

A JavaScript IDE that the front-end relied upon for developing various TypeScript and JavaScript components, known to be among the best in the industry for its rapid code analysis, refactoring, and generation tools.

15


II.1.3.6 Testing tools

Postman

A cross-platform suite for testing HTTP related software. Though not the most advanced testing tool for this purpose, the simple user interface, excellent featureset. Throughout the project it was heavily used for integration testing the logging service and the feed service.

II.2 Cooperation agreement In mid November 2016, the group members entered into a cooperation agreement for doing the Infront Business Intelligence bachelor project, with the specifications that they would both spend between two and four days a week working on the project, primarily at Infront's offices where space and resources were allocated for the group. The division of labour would be split in such a way that the two parties would be able to work effectively and independently on the actual coding aspects of the project, minimizing overlap in work and increasing productivity. This still allowed for frequent discussions and re-evaluations, as the project's prototype nature required. Eirik Lunnan Djuve would be student lead for the project, and primarily work on the front-end, as he was accustomed to from his part time job at Infront, and Simen Dagfinrud would work on the back-end. Both parties would be heavily involved in writing documentation, reports, and any meetings with supervisors to get advice and guidance on the project’s direction and current status.

II.3 Development

II.3.1 Research Phase Following the group's formation and the decision being made that the project would be Infront Business Intelligence, the group spent considerable time researching the various technologies that could be used. Both members of the group agreed that the project should attempt to utilize new, open source technologies wherever possible in the project, as this maximized learning potential and allowed better control of the project's outcome. Eirik spent most of this time researching Angular and single page applications, whereas Simen read up on PostgreSQL and database optimizations, as the languages the project would be written in were not yet set at this point.

II.3.2 Planning at Infront In the early stages of the project, the group had numerous meetings with the client supervisor and other developers to figure out possible ways of implementing the Business Intelligence solution.

16


Eirik had a higher level of familiarity with the terminology due to experience working in the company. Coming from outside the company, Simen had to become familiar with the structure and flow of information in Infront’s systems as well as new terminology, both for Infront’s own systems and for the domain of financial trading. Initially, languages considered for the back-end were Python, R, Java and C#, but it quickly became apparent that a project of this scale required as little time spent on integration between various languages and frameworks as possible in order to successfully reach the goals set. The client supervisor strongly advised using C# as the primary programming language for the back-end, as it would allow more developers at Infront to assist the group should it be necessary. The front-end was bound more or less to the conventions of Infront’s web development team in using JavaScript and TypeScript for actual web development. In addition, Angular uses TypeScript, making these the only rational choices for developing the SPA. The back-end and front-end both decided to use ASP.NET Core and .NET Core for hosting the applications. This proved useful, as the technologies were bleeding edge at the time, lacking proper documentation and tutorials -- with both group members familiarizing themselves with the same new technology, sharing tips and advice greatly aided getting the hosting solutions running. Following advice from the client supervisor, the group started out creating various sketches and mockups of user interface and data flow. The deadline for the pre-project was coming up, and the group agreed that it was best to postpone any coding until there was a concrete plan and less vague requirements for the task at hand. It was decided that the project report, and the code, would be written in English, as that was the standard in Infront as well as increasingly common in software companies throughout Norway. The HiOA supervisor recommended that the group keep a small project diary and a site for the project, and that they had weekly or semi-weekly meetings with him, either remotely, at the school, or at Infront’s offices. The early meetings helped shape the project and ensured that the group remained on track. There were numerous problems initially in writing the pre-project report. Bugs in Microsoft software at the time made cooperation unstable, resulting in lost work and constant annoyances while editing the same document. Subsequently, LaTeX in combination with Git for source control of the report was suggested, though quickly struck down due to its complexity and steep learning curve for the uninitiated. Finally, Google Docs was chosen as the primary tool for writing the report, and work on the report accelerated quickly as the deadline approached by allowing a shared platform for storing and editing files related to the project documentation and reporting.

17


II.3.3 Back-end The back-end development started out with the log service per the client supervisor’s recommendation. As Simen lacked familiarity with the Microsoft Web API and MVC frameworks, the early stages of the project largely involved testing out various ways ASP.NET Core could be used for achieving the project’s goals. Initially, to avoid spending too much time on writing queries that might not be used, an in-memory database served as the storage layer in the back-end. Due to Entity Framework Core’s highly modular and dynamic functionality, it would allow the vast majority of code written to be re-used upon setting up an actual database. Entity Framework Core is a so-called ORM or “Object-relational mapper”, which allows for a practice called Code First, where one models the database in object classes. The database is then specified through a database connection string, and generated by using pre-existing libraries that translate code into whatever database language is required. Further, Entity Framework Core allows for altering data models through migrations, which would prove extremely useful as the project progressed, and the models had to be re-defined to become well optimized and integrated. There were however hurdles in dealing with various edge cases, and debugging problems due to Entity Framework Core being a significantly less mature technology than its proprietary parent, Entity Framework. A very simplified front-end, using the Razor templating engine and simple MVC Core controllers served as a testing platform for demonstrating that data was stored and retrieved correctly. This was later abandoned entirely as the project’s scale and complexity grew. After there was a certain familiarity with C# and ASP.NET Core, work began on setting up a Web API to receive and handle data from the WebTrader. An issue with this was that there had been minimal interaction with the WebTrader, and as such, data was mocked rather than retrieved. The actual form of the data was unknown, as the logging component had yet to be written. The project's fluid, prototype nature required continuous testing and re-testing of large portions of the code-base, as well as testing new database models. Extensive unit and integration testing was required throughout the development process to ensure that there were no cascading errors leading to confusion and hindering progress. Thankfully, the client supervisor was readily available to answer questions through Skype for Business or in person when they arose. Postman was an invaluable tool in testing the Web API quickly and effectively, and could even handle minor load testing by sending many requests per second to the log service. For ensuring regular transmission of data to the feed service provided by Infront for parsing our Business Intelligence data, some kind of scheduling class or application was required. Attempts

18


were made to integrate a timer in the Web API itself, but cursory research indicated that it would be more stable to separate scheduling entirely from the log service to ensure stability and accuracy. A .NET Core console application was written, using simple methods that executed HTTP POSTs that would trigger any code necessary in the log service itself. Unit tests for the timing itself, and later, integration tests were run regularly to ensure that the system worked as it should. Being unfamiliar with the WebTrader from the outset meant, to some degree, exploratory tests were required to find out what exactly could be collected from where, and how to best integrate this with minimal performance loss. It was therefore necessary to not only unit test utility classes for handling and formatting the request data received, but also that the data itself was accurate and reflected by the logging suite on the WebTrader. Integration tests were run after any change to a utility function or a logging function so that the results could be verified and errors discovered quickly. Postman was used to ensure that the end-point for a HTTP POST actually could receive data prior to actual integration tests. Testing paths were set up in the ASP.NET Core logging service that would give an indication of whether the code had been received, and if it had been received, whether it contained the correct data. As more and more pieces of the back-end came together, and logging was functional, it became possible to create count-analyses from the data using queries run on optimized, fairly denormalized SQL tables (no higher than 2NF to avoid necessitating joins and reducing the impact of high performance indexing). The last thing to be implemented was the logging and counting of symbol usage, as it required dynamic generation of tickers. Several workarounds were implemented to get Entity Framework Core to obtain raw count data from the log tables, as it did not support dynamic model creation in the code at the time of the project.

II.3.4 Front-end As the front-end relies heavily on integrating Infront's web-toolkit, integration testing has been prioritized before unit testing. An important motivation for creating a single-page application was to test how the newly developed web-toolkit would perform in such an environment. Refreshing the page, an action performed frequently in a multi-page application, takes care of things like closing open websockets, clearing JavaScript variables etc. How the toolkit would behave in an application that by design should never have to do a complete server refresh was unknown. As fixing any bugs would largely be dependent on client resources, it was important to find and report bugs as early as possible. To do this the group chose to continuously conduct error-guessing and exploratory integration testing while developing the application. Bugs were

19


documented and reported through the appropriate channels (mainly trello-boards) immediately, and re-tested when fixed. The tests were conducted manually, by troubleshooting any visible errors in Chrome Developer Tools, and monitoring websocket traffic. The group selected these test methods as they require little preparation time, and allowed focus on errors predicted by the client. Fully testing the toolkit using a more formal test method was not deemed suitable for the purpose of this assignment, as it would be very time consuming. The main drawback of this type of testing is the lack of documentation, sometimes causing difficulty in showing exactly which test has been run, and which steps lead to the error(s) in question.

20


III Product Documentation



21


III.1 Requirements

III.1.1 Initial requirements The following requirements were specified in the pre-project phase:

III.1.1.1 Must ● The data acquisition and analysis component of the project must extract and analyze

information about user activity with regard to time and how the user interacts with the WebTrader. It is essential that all data can be mapped to both user and provider for the data to be useful not only to Infront, but to their customers as well.

● The project must result in a working web application, using the real data produced by data mining and displaying the processed result live.

III.1.1.2 Should ● The project should collect and analyze information about what stock exchange

information the end-users utilize (actually look at).

● The project should be a single-page app implementing the Angular 2 front-end framework.

III.1.1.3 Could ● The project could extract information from searches within the WebTrader, specifically

the query and the associated chosen result to improve the search quality. It could also implement single sign on for accessing the analysis server/service.

● The project could implement Infront's single sign-on solution.

In addition to the listed requirements, there were some required Infront-specific dependencies. The front-end project was to implement Infront's web-toolkit, and generated data would have to be submitted using architecture meant for providing market data. These initial requirements were lettered by the group members, based on our understanding of what the client wanted to achieve. They acted as the primary conditions for the software requirement specification, found in the appendices.

III.1.2 Additional requirements Different strategies for storing, parsing and processing data was discussed in the pre-project. As the main source for usage data will be the WebTrader, it became apparent that the simplest way to transmit data would be through HTTP-requests. The client uses ASP.NET for all of their WEB-applications, and required that web applications created as a part of the assignment would

22


use the same framework. To futureproof the solution for platform independent release, the client wanted us to use the newly developed ASP.NET Core. As the front-end solution showed progress, proving the toolkit worked well as a single page application, the client also wanted to explore how this could be tailored for mobile platforms. Included in this requirement was researching frameworks for web-applications, providing the look and feel of native applications.

III.2 Dependencies

III.2.1 ASP.NET Core To serve as a framework for both the back-end and the front-end, the project utilizes a relatively new technology by Microsoft, ASP.NET Core. It is an open source version of the much-used and well known ASP.NET framework that aims to have cross-platform compatibility. Currently it implements large parts of ASP.NET’s functionality, but is lacking in certain areas. ASP.NET Core allows the project to set up high-performance web applications and APIs using Kestrel as its web server. For the purposes of the project, IIS serves as the middleware between the application and the web-server, though this can be customized through OWIN or a reverse proxy server. Entity Framework Core is used as a layer ASP.NET Core and a compatible database, allowing for Code First database modelling, and repository/abstraction layers should they be necessary. Code First and the ability to perform data migrations and on-the-fly model updates allows for a highly fluid and adaptable workflow for the back-end. ASP.NET Core contains the ASP.NET Core MVC framework, which merges MVC, Web API, and Web Pages with the Razor templating engine, allowing for rapid development and testing of web applications. Minor aspects of the MVC framework were used for the back-end initially, though largely phased out as it became a RESTful Web API. Major highlights of the Core MVC framework are customized routing (allowing for various methods to be executed upon POSTing data to different URIs), dependency injection with transient, scoped, and singleton patterns, and highly optimized asynchronous handling of requests.

III.2.2 Infront web-toolkit To access and display the collected usage data, the project makes use of Infront's web-toolkit. The toolkit is developed and distributed by Infront to enable their institutional clients to include market data and trading, hosted by Infront, in their web applications. Infront's web-applications are also developed using this toolkit. It is written in pure JavaScript, using TypeScript, a typed superset of JavaScript that compiles into plain JavaScript.

23


Infront Web-Toolkit follows the Model-View-ViewModel (MVVM) - pattern, where all values displayed are the result of data-bindings between the view and the viewmodel, that again is based on the actual model. Knockout.js is a well known JavaScript framework, following the same pattern. For data transportation, the toolkit facilitates full-duplex communication between the browser and Infront's WebSocket servers. This connection is used both to provide historical data, and to supply real time data. The WebSocket API is used as it supports event driven responses, meaning updates can be received without having to poll the server. This is crucial for applications meant to present real time data, as the alternative would be constantly polling the server to check for updates. This layer, handling data connections and distribution, acts as the data model. The toolkit supports ways of presenting data, each organized as a "widget". Web widgets are defined as small applications that can be executed within a web page. To date, the toolkit includes 22 widgets for market data presentation. Our solution only utilize a handful, deemed useful for our application. Each of the widgets, and their original use is further described in figure 3. Each widget represents a Viewmodel, including all exposed methods allowing interacting with the view. As all views are injected, they are also responsible for creating views and their data bindings.

Widget Name Use

Chain Viewer Widget used to show pre-defined lists, called chains. Chains are typically a list of the companies (constituents), included in an index. Supports showing multiple parameters, typically OHLC and timestamp of last trade.

Chart Widget used to graphically present symbols price development over time. Can display multiple symbols to compare their performance.

Focus Widget used to present a quick overview of a single symbols current performance. Shows the current price, price difference since the start of the day (absolute value and percentage), and a small graph showing how the current price relates to today's high and low.

Index Overview Similar to focus widget, but only used for indices. Small graph shows the performance of underlying constituents, and table shows how the index has developed during different periods of time.

Market List Developed by Eirik as a part of this project, to fill missing functionality. Further described in the product documentation.

24


Already included in the toolkit, as it can be used to display all markets available to the current user.

Quote List Widget used to show a list of all stocks available on a specific market. Supports showing multiple parameters, typically OHLC and timestamp of last trade.

Ranking Widget used to present a list of the stocks with the most extreme change in either price or turnover. Can display either "winners" - the symbols that has increased to most in value during a period of time, "losers" having decreased the most and "turnover" - showing the instruments with the highest amount of shares traded during the period.

Search Simple plain text search, to find specific symbols.

Valuepair Simple single value widget, displaying two supplied parameters for a given symbol.

Figure 3: Infront Web-Toolkit widget overview

III.2.3 Angular 2 To create a single page application, the front-end solution depends on Angular 2. Angular 2 is a web-application platform/framework, developed by the Angular team at Google. The platform is made specifically for creating single page applications (SPA), and is based on Typescript. Developing applications using Angular 2 is conducted using Typescript, HTML and angular specific HTML-elements called NG-elements. The source code is compiled to JavaScript, runnable in any modern browser (polyfills needed to support older versions).

25


Figure 4: Angular 2 high level architecture. Retrieved from www.angular.io, in accordance to Creative Commons BY 4.0 licensing. Angular applications are built by splitting and organizing content into components. Each component can control a patch of the screen called a view. Simply put, a component equals a JavaScript class, following an angular specific interface, instantiable by Angular's loader. Metadata is attached to the class to tell Angular that it is a component, and configuration parameters Angular needs to instantiate it. The most useful parameters include:

● Selector: the CSS selector to be associated with component

● Template(Url): parameter telling Angular which template to use

● Providers: array of dependency injection services that should be injected at instantiation

The layout for each component is defined by its Template, a form of HTML that tells Angular how to render the component. Templates uses typical HTML elements, in addition to NG-elements and directives. NG-elements and directives adds dynamic behaviour to the template. Forcing content to be rendered and updated relative to variables in the component, acting as data-model for the created view. Angular includes its own dependency injection framework. Passing a usable object (a service) to supply another object's dependencies helps decoupling objects. Decoupling objects is desirable, as it allows updating an object, without having to change all of the objects it depends on. Splitting objects can also result in more reusable code (Jenkov, 2015). Services injected as dependencies in Angular are singletons, meaning single instances of the service can be shared by multiple object's. This is useful for persisting state, and can enable parent and children components to communicate. Angular makes it possible to create single-page applications by including its own router, enabling navigation between views based on user input. The router is configured to associate a specific component to a URL. Angular's router will interpret a browser URL as an instruction to render the associated component, all though it will not prevent the browser's server call. To navigate without server communication, components can send server-side instructions to the navigator. The navigator will then update the relevant content of the page, without having to reload the page. The navigator is implemented as a dependency injected service. For further Angular 2 specific reading, please see the referenced Angular documentation.

III.2.4 Onsen-UI To create views for mobile devices, with the look and feel of both iOS and Android devices, the group as selected to implement the Onsen UI framework. The framework offers UI components

26

http://www.angular.io/


and routing customized for applications using the Angular 2 framework, making it highly suitable for our application. Using Onsen specific syntax/elements in the Angular templates compiles to HTML5 elements with styling similar to elements native on the target device. The framework includes logic for identifying what device the application is rendered on, automatically including the relevant device-specific styling.(Onsen UI, n.d.) Onsen-UI also includes its own router. This router differs from the Angular 2 router by pushing new views above the current view, creating a navigation stack, rather than updating the same view. This more closely resembles native applications as it allows a similar transition between views. Views further down in the stack are simply hidden, meaning switching back to the previous views can be performed instantly.

III.3 Product description The product is a prototype of a complete Business Intelligence system for the Infront WebTrader, including logging on the Infront trader, a log service for handling the data and passing it on to allow integration, and a single page application integrated with existing Infront libraries and systems to present the data using advanced data presentation widgets.

III.3.1 Data Figuring out how to structure and organize data in an efficient and functional way has been a key part of this project. The group has opted to group data by provider ID (PID), the number stated in every user-profile representing which provider they are associated with. The PID is always unique to one provider, but each provider can have multiple PIDs. Usually to differentiate which country the customer originates from, for international providers.This abstraction was selected as it is the least common multiple available, in addition to believing it would be useful both to Infront and their providers to be able to differentiate results e.g. on geographical origin. Logging was constrained to whatever data could be accessed server side. Mainly being logins, page access/requests, and heartbeats. Heartbeats are periodical requests sent to the server, to maintain the user's session in accordance with the HTTP protocol (IETF, 1999). This could be used as an activity measure. The amount of heartbeats received within a time interval would be strongly linked to the number of users with an active connection in that period. The logging implemented in the client application can produce a huge amount of measurements. Theoretically one measurement for every stock and market available, multiplied by the number of PIDs associated with the webtrader. Each measurement translates to content that has to be accessible in the application presenting the data. In addition new entries could be added continuously, as entries will be added each time a stock is accessed for the first time.

27


The content available also has to be relative to the users credentials, as providers should only be allowed access to data about their own customers. As a result the application is developed to be highly dynamic. It would be extremely time consuming to develop and maintain an application with any static references. As the data must be made available through a system made for market data, typical tools such as metadata and structure was highly constricted. The group opted to organize data by PID by bundling them to a feed. A feed is the data-grouping used by Infront for organizing data from a specific market under one feed. Access to market data is managed by allowing access to a feed. Organizing BI-data in the same fashion allows managing access in the same way. Authorization will be given using the same tools as when allowing access to a market.

III.3.2 Logging The logging aspect of the product takes place entirely on Infront’s proprietary WebTrader; a web-application. It obtains usage data either from a response to a request it sends (heartbeat), or requests sent to the web-application containing session and page information. The data gathered is unrelated to trading to avoid any legal issues with trading regulations, and usernames are anonymized with a strong cryptographic function so as to keep data confidential should there be a breach of security.

Figure 5: Test data taken from staging server, single user only

28


III.3.3 Back-end

Figure 6: Activity diagram of logging a generic request

29


Figure 7: Activity diagram giving overview of analysis and transport The back-end of the solution is a service that asynchronously receives logs in the form of HTTP POSTs, parses the data, asynchronously stores it in a relational database, analyzes the data, parses the analyzed data into a specific format, and then passes that formatted data on to be parsed into the proprietary format that Infront uses for handling stock and trade data. It has a scheduling component that is presently a stand-alone application to prevent thread interactions and problems under load. The code for the scheduling component is simple enough that a developer could easily understand and alter the code to separate the functionality out into the Windows Task Scheduler on Windows, or Cron on Linux, allowing for native-server task scheduling. Upon receiving a log, the back-end parses this into an object that is subsequently stored in the database. The technology used as a middle-layer between the back-end’s service and its database allows for switching out the database with another with relatively few modifications to the code, should there be a need to use different technology (this may include changing some of the SQL used as well).

30


Presently, the analyses being performed are database-query based, but can easily be extended to include more advanced statistical and numerical analyses through using independent components for interacting with the database and creating their own tables to be accessed by the log service at specific intervals. The database is implemented through a Code First approach, allowing a model to be changed on the fly should it be necessary. This aids in making a highly adaptable code-base for the prototype BI back-end. To ensure that the technology is reasonably future-proof and adaptable, all of the back-end was written using cross-platform, open-source technologies. Constraints involving the use of Entity Framework Core, the open source version of Entity Framework, were a lack of functionality and support for inheritance. Figure 22 shows the actual form the tables took. Figure 23 shows the form they ideally would have had, if the functionality was implemented in Entity Framework Core. The interactions into, and out of the back-end are all in the form of HTTP POSTs, allowing for simplicity and adaptability to changes, while simultaneously being reasonably tolerant to heavy load on the server and asynchronous handling of data. Though the project has its own front-end component, the data gathered and analysed in the back-end can also seamlessly be presented in the existing Infront Terminal, the desktop application and main product of the client.

III.3.4 Front-end Infront BI's front-end presents the distributed BI-data to authorized users, using Infront's web-toolkit, abbreviated WTK. The application is "single-page", meaning content navigation is conducted without loading another page from the server. The content of the page is updated dynamically, more resembling a desktop application than traditional, multi-page, web applications. This is application is made single-page by using the Angular 2 framework. Using this framework, the page is split into components, enabling targeting and updating only that specific part of the screen. The content of each part is called a "view". The term "page" will be used for views updating most of the screen, even though every one of them is rendered on the same page. The examples shown in this section will either be empty, or filled with mock data. Most of the data presentation widgets requires that data has been collected continuously for some time to fully demonstrate their functionality. Producing data also requires that someone actually use the Webtrader deployed in the test-environment. This has not been possible in time between deployment and this documentation being written.

31


Desktop application The application is structured to follow the same hierarchy as the underlying data, resulting in three hierarchical levels; provider, PID and "symbol".

Figure 8: Top-level page The top level, serving as the application's home page, is intended to present an overview of the data with the highest level of abstraction available to the user. The application is intended for two types of users; Infront administrators and provider administrators. For provider administrators, this top-level page will be an overview of data concerning all of their customers. Regardless of the customer's geographical location, subscription etc. For Infront administrators, the data will concern all users regardless of provider. The data this view is intended for does not exist at this time. The PIDs, i.e. the feeds, available to the user will be listed in the column on the right hand side of the page. This data is requested and displayed using the market list widget. This column is implemented as a separate component, and does not reload when updating the column(s) to its left. The lists show the actual BI-feeds added to Infront's systems for the purpose of this project.

32


Figure 9: Second level page (demoed using Oslo stock exchange data) The second level pages will present an overview of the data with the lowest level of abstraction. For provider administrators this will be data concerning all users related to a specific PID, and for Infront providers it will present data grouped by both provider and PID.

Figure 10 - Single metric page showing PID 1 logins (mocked/generated data) The third level pages will present the data available for a single metric. Clicking on a metric in a list, or selecting it from a search result will navigate the user to this page.

33


Mobile application The front-end solution includes pages optimized for mobile devices. These pages combine the benefits of single-page applications with user interface, navigation and transitions similar to native applications. The result webapplication gets a look and feel closely resembling fully native apps . The Onsen UI framework identifies the user's device at runtime, and positions and style elements to resemble the target operation system.

Figure 11: Mobile view "global overview" (iPhone left, Android right) As BI-data was not available during the development of these pages, they are customised to show regular market data. These views use the same components and services as the BI intended desktop, but have a layout optimized for smaller screens. The mobile version follows the same three layer hierarchy, only the content is split into multiple tabs. The application is configured to automatically navigate to the mobile pages when displayed on a mobile device.

34


The bottom menu present on the iPhone screenshots has been configured to only be included on iOS devices. The menu button is rather placed in the top left corner. This follows the conventions of native apps, familiar to each platform's users.

Figure 12: Mobile view "market overview" (iPhone left, Android right)

III.4 Implementation

III.4.1 Backend

III.4.1 Logging As the logging had to be injected into existing WebTrader code, it was written in C# as that is the language the WebTrader is written in. Most of the methods used were placed into a base controller class, allowing them to be called from the various controllers and filters necessary to collect the data required. Guidance from the client supervisor was imperative to allow as much existing code as possible to be used in the logging aspect of the project, minimizing the impact the Business Intelligence project would have on WebTrader source code in the event of future merges or integrations.

35


The logging was injected into controllers for heartbeats, markets, and logins. Symbols required a more advanced approach, using filters to identify the data in the request and acquiring information about which symbols users accessed. SHA256 was used to anonymise usernames in case of a data breach, and to stay in compliance with financial and privacy laws.

III.4.2 2 Log Service The log service implements RESTful web APIs to allow easy integration between the various models. Its controllers activate all code in the project, relying on advanced dependency injection (either transient, scoped or singleton), and Core MVC’s routing modules for simplicity and elegance. The Scheduler is a stand-alone .NET application that executes HTTP POSTs to specific URIs on the Log Service.

Figure 13: Example code from Scheduler

36


Figure 14: Example code from Receive controller Through the Receive controller, the application executes code to store information in the database asynchronously.

37


Figure 15: Example code from the Send controller The Send controller receives commands from the Scheduler and activates code depending on which URI was POSTed to.

Figure 16: Example code from an utility class

38


Figure 17: Example code from the Time utility class

Angular Modules The front-end solutions top level building blocks are angular modules. They are classes with angular decorators, that in the same way as angular component decorators tells angular how to load the module. The modules function as container objects, designed to store multiple instances of other components. Our service providers are added to the module, making it possible to hold a singleton instance injectable to all contained class instances.

Figure 18: "InfrontModule" Angular module provider declaration and exports Exporting components from the module makes them accessible by other modules, without having to declare them in each module. Other modules simply have to import the module to access the exported components. Services and components directly utilizing Infront's web-toolkit is organized as a "infront module", separating the highly Infront-dependant, non application specific code. Such a module

39


could be included in the web-toolkit. Making it easier for customers to include the toolkit in their own Angular 2 applications. The module includes all widgets implemented in angular, as well as services for simplifying communicating with the toolkit. Among them user authorisation/login, cookie storage/retrieval, security service for navigator etc. The desktop and mobility views and navigation services are split into two separate modules. Both import the infront module, but are dependant of each other. The desktop module simply imports the mobility module to be able to navigate to it when it detects a mobile device.

Components

Figure 19: Market List widget component Figure 19 shows the code for the Market List widget component. The first line imports a reference to interfaces included in Angular 2's core module:

● Component: The decorator needed to tell Angular that the "MarketList" class is a Angular component.

● Input: Decorator used to declare a variable name accessible to bind to a variable from the parent component when instantiated.

40


● OnChanges: Angular life-cycle hook. Method is executed each time a input variable is changed.

● Elementref: The type of references added to the template. ● ViewChild: Decorator used to create the reference ● OnDestroy: Angular life-cycle hook. Method is executed when the component is

destroyed. The second line imports a reference to our "InfrontUIService" used to hold the infront object returned by the web-toolkit as a response to a successful login. The component takes an option object as input, and destroys and creates a new widget if the options object is changed. The template HTML-element part of the reference is appended to the widget constructor, that injects the widget into the referenced element. The widget is destroyed properly when the the component is destroyed. All widget components follow this general recipe. Other components are more or less similar, but typically have more complex templates. We have written more complex templates in separate HTML-files, and reference them using require. The most frequently used life-cycle hook, "OnInit" is not included in this example. Functions using the ngOnInit decorator are executed once, when the component is instantiated.

Services

Figure 20: Excerpt from "MarketService"

41


Injectable services are decorated using the @Injectable decorator. We have mainly used services for persisting state and communicate between components and modules. The reference path at the top of the excerpt in Figure 20 makes sure the declarations file for Infront's web-toolkit is evaluated before the service (and any component injecting it). Not doing so would result in a compilation error, as any type references would fail.

Mobile templates

Figure 21: Excerpt from "spaHome" Angular2-OnsenUI template Figure 21 shows most of the markup in the "spaHome" template included in our mobility module. The CSS selectors starting with "ons" is parsed by OnsenUI to inject HTML-components with native resembling styling. The "ons-if" tag injects conditionals, resolved at runtime to decide whether or not to include the elements between the start and end tag. Here, the conditional checks the content of the variable "platform". This variable is supplied and resolved by the Onsen UI framework, stating which platform the application is rendered on. "Ons-icon" takes multiple icon parameters, where the parameter with the material key is selected when displayed on an Android device. The "ons-tabbar" renders a complex multi-tab element, where all child elements are rendered, but only the active one is visible. The "tab1","tab2" etc. references is declared in the component that loads the template.

42


Webpack The modules are bundled using webpack. A javascript module bundling tool. The entire codebase is compiled and bundled into one javascript file. Webpack also includes development middleware, allowing hot module replacement. This enables automatically updating the bundle when updating the codebase (webpack, n.d.). The webpack configuration, startup.cs and project.json is based on "ASP.NET Core Angular2 Starter Application", included in ASP.NET Core Template Pack. The template pack, created by Mads Kristensen, is available as an extension for Visual Studio. (ASP.NET Core Template Pack, 2016)

III.4.4 Design As the application is intended as a companion for Infront's Webtrader, the client has asked us to use the same visual and interaction design. This includes colours, layout, sizing and navigation paradigms. To create mobile layout resembling native applications, the group has integrated Onsen UI with Angular. Onsen UI ships with its own stylesheets, and the group has opted to mainly use its default styling. As the application does not target the general population, it is not directly affected by regulations regarding universal design. The purpose of this assignment has been to create a prototype to evaluate technological solutions, meaning resources spent on visual design has been kept at a minimal level. Regardless, effort has been made to offer sufficient usability and user experience.

III.4.5 Database design The database design is a result of rapidly changing requirements and conditions, made necessary by the project’s “prototype”-nature. The DBMS chosen was PostgreSQL, an object-based, relational database, for various reasons -- one of them being that the framework .NET Core supported it natively, and that PostgreSQL is known to be advanced and customizable as well as open source. Being that the project had a relational database, it was necessary to ensure that it allowed for a large number of asynchronous inserts into a variety of tables through allowing tables to be denormalized enough that most queries could be executed simply and with as few joins as possible. It ended up being on 2NF to allow “non-relational” speeds on a relational database. Microsoft Entity Framework Core, a Code First frameworkj, was chosen for modelling and migrating the database

43


Figure 22: Structure of Entity Framework Core classes for Code First database generation (actual)

Figure 23: Entity Framework version of Figure 22 (not used due to lack of functionality)

44


III.5 Result

III.5.1 Client feedback “The students have shown an excellent ability to learn new programing languages and adapt to pre-existing systems and architecture within the timeframe of this project. During their time with us they have created a prototype that shows how we can implement business intelligence tools and integrate them with our own systems in a very efficient way. While integrating against our systems they also helped us uncover several issues with some of our existing software and highlighted potential future issues with scalability. The project has been an important resource for planning future development of our web-based products and we are very pleased with the overall results.”

- Ole-Martin Grøtterud, client supervisor (23.05.2017)

III.5.2 Discussion The Infront BI project has resulted in a fully functioning single-page application, including mobile friendly views, implementing Infront's web-toolkit. We have also, at least to some extent demonstrated the feasibility of transporting business intelligence related data using Infront's solutions for market data. Revisiting the software requirement specification, the group believes we have completed all requirements listed as 'must' and most of the 'should' requirements. Shortcomings are mainly related to missing data. Difficulties with unfamiliar frameworks and coding languages, coupled with changed and/or different understandings of requirements has prevented us from reaching our goals for computing data. What we have done, is demonstrating that it is possible. We are also pleased with the services involved in producing what data we have, as they have proven to by reliable and efficient. The one-to-one relationship between provider ID and feed also seems to be the right decision. The lack of BI data to present in the web-application also means that it is hard to judge how successful we have been to create an application for visualizing BI-data. We are pleased with figuring out that the volume parameter can be used to accurately presenting counts period. Important, as charts would not have been usable without it. In regard to data visualisation, the way of calculating price over time has proven the most challenging. Looking back, we would probably not have opted to use the newly released .NET Core for anything but the front end solution. Especially issues with entity framework has been to time consuming. The extensive use of frameworks and libraries in the front-end solution has provided us with lots of great features, but has also caused a lot of dependency issues. Finding a combination of versions of all node modules has in no way been trivial. Some of the libraries

45


force us to use older versions of other, more frequently updated libraries etc. Some issues have even forced us to implement community developed workarounds. We are probably most pleased with our development of a single-page application, successfully integrating both Infront's web-toolkit, Angular 2 and Onsen UI. The group helped uncover several issues, both important for today's applications and future development. The client has started to develop a hybrid web-app for mobile platforms, based on the work conducted in this project.

46


IV Appendices



47


IV.1 Financial technology A major part of this project has been to explore possibilities for transporting, storing and representing BI-data, using existing architecture. The existing architecture is tailored for processing information from stock markets, using a specific format. This section will present background information, terminology and acronyms required for understanding the domain specific topics addressed.

Markets "Market" is the term used to represent "where" a stock is being traded. This can either be formal exchanges; a market with a central physical location, or over-the-counter (OTC) markets without a specific geographical location.

Stock trading Stocks are exchanged all over the world through exchanges and other markets. Equities; stocks of publicly held companies, is probably what most people think of when hearing the word "stock". Buying equities gives the buyer ownership in a corporation, and represents a claim on a part of the corporation's assets and earning. When a company is listed on an exchange for the first time, commonly called "going public", equity value is based on what is considered to be the company's value. The company's value is calculated, simply put, as the company's assets subtracted the company's liabilities. The price per stock is then relative to the number of stocks being issued. These stocks are then sold on the open market, raising money to the company being listed. (Wilmott, 2007, p. 2) After the initial public offering (IPO), price is mainly governed by sellers and buyers. To be able to buy a stock on the open market, someone already owning the stock has to be selling it, and be willing to sell it for the price offered. To be able to make an informed decision on when and at what price to trade, traders rely on information about what the stock has been traded for in the past, and what prices others are offering to sell and buy the stock. Infront provides solutions providing this information, including real-time data, historical data, analysis and trading. Stocks/equities are perhaps the most well known, but there are many other types of financial instruments traded on the open market. They represent different types of financial values, but are traded on the open market in similar ways.

Indices Stock Market indices are non-tradable stocks created to measure how a market is doing as a whole. A typical index is calculated from the weighted sum of the prices for a selection of stocks, deemed as representative for the market. (Wilmott, 2007, p.11) Dow Jones Industrial Average is among the most well known. Indices are also constructed to reflect a specific sector of the

48


economy, i.e. the Dow Jones Oil and Gas Titans 30 Index that is made up from a pre-weighted average of the 30 oil and gas company listed at Dow Jones deemed most important.

Stockbrokers Stock trading done by non-professionals is mediated through a stockbroker. A professional registered representative, licensed to execute buy and sell orders through a stock market. No further explanation of trading will be given in this report, as technology used for trading has not been deemed suitable for the purpose of this project. Further background for this decision can be seen in the process description (p-xx-xx)

Ticker symbol To identify a stock, the market it is listed on issues a 1-5 letter identifier called "ticker symbol", commonly abbreviated "ticker". The ticker is unique only to the market the stock is sold at, meaning the ticker can be issued to another company in another market(or the same company, if it is listed on multiple markets). "STL" is the ticker for Statoil at the Oslo Stock Exchange (OSS), but is used for Sterling Bancorp on the New York Stock Exchange (NYSE).

Market data representation A previously concluded trade is represented by three values; the precise time when the trade was executed, the price per stock, and the number of stocks exchanged in the trade. These values are commonly called "time", "last (price)", and "volume". When looking at a stock's performance over time, viewing one trade at the time is highly impractical as there can be thousands of trades per minute. Therefore it is common to supply historical data grouped by time periods, ranging from per second, to per month or even year. The time period the data is grouped by is commonly referred to as "resolution", i.e. viewing data grouped by 5 minute intervals in 5 minute resolution. Resolutions below 1 day (24 hours) are commonly referred to as "intraday", and any others as "historical". To represent a period, the stock price is represented by four values; the first price, the highest price, the lowest price and the last price the stock was traded for during the period. These values are commonly called "open", "high", "low" and "close", abbreviated "OHLC". The period gets timestamped with the start time of the period, and volume represents the total amount of stocks traded. If a stock has been traded only one time during the time period, or multiple times at the same price, open, high, low and close would be the same number.

Trade history stock "x", on market "y" from 10am (1 min resolution)

Period variable Open Low High Close

49


Timestamp 10.00.01 10.00.08 10.00.30 10.00.41 10.00.59

Price 120.1 120.0 120.3 120.7 120.4

Volume 200 500 800 500 1000

Figure 24: Trade history example The example data shown in Figure 24 could be represented by the string; "ticker: x, market: y, time: 10.00.00 , open: 120.1, high: 120.7, low: 120.0, close: 120.4, volume: 3000. Please note that volume is the only aggregated variable, as aggregating price would not result in any meaningful information. When it is only possible to show one value, i.e. in a single line chart, the default setting is normally to show the last price. The average price can be meaningful, if volume per trade is taken into account. This value is called "Volume Weighted Average Price"(VWAP), and is calculated using the equation shown in Formula 1. It is normally only available for "non-intraday" resolutions, partly due to the amount of processing needed to calculate the variable continuously.

wap v = Total shares boughtΣShares bought x Share price

Formula 1: Volume Weighted Average Price Calculation

Market data distribution Infront is neither a stock exchange or a stockbroker, but a market data provider and trading platform. Meaning their role is to provide data from multiple stock markets to professionals and non professional stock traders, and solutions to trade stocks. The latter includes both solutions for placing bids and sell orders, and solutions for the traders executing them on the specific market. A simplified high-level visualization of this data flow can be seen in Figure 25. The trade information transmitted by the stock markets consists of one entry per transaction. Each transaction includes ticker, timestamp, price, and how many shares was traded in the transaction. Infront's systems then formats this data to fit their data-model, forwards the data to their applications in milliseconds, and use it to calculate intraday and historical resolutions.

50


Figure 25: Infront market data flow visualization

51


IV.2 Software requirement specification

Introduction The purpose of this document is to give a detailed description of the requirements for "Infront BI". The document is primarily intended to document required functionality, constraints and dependencies agreed upon by the client and students.

Scope "Infront BI" is a Business Intelligence tool, providing relevant information to Infront and their providers. The information is made available through a single-page web-application, also available on mobile platforms. Understanding how users utilize Infront's products is crucial to help Infront develop the solutions the user's need. Providing this information to Infront's providers is also important to allow them to experience first-handedly how their customers react to changes, and to tailor solutions fitting their needs. "Infront BI" is a prototype, and is not expected to appear as a finished product. It's purpose is to examine the feasibility of developing a product within the stated goals and conditions.

Definitions

Term Definition

PID Abbreviation for Provider ID. All WebTrader users are uniquely associated with a PID, identifying which of Infront's providers they gain access through. This in turn determines what feeds the users have available, which is how access to Infront Business Intelligence could be given both at “provider level” for providers, and a total overview level for Infront developers. Providers often have multiple PIDs, frequently related to operations or offices in different countries to allow customization of data packages for the end-users on a country to country basis.

BI Abbreviation for Business Intelligence, actionable information acquired through a technology-driven process to help make more informed business decisions.

Overall description This section will give an overview of the whole system. It will describe the components needed to develop the solution, their basic functionality, and how the components will interact with other systems. Constraints and assumptions for the system will be presented, along with a description of stakeholders and how they will use the system.

52


Product perspective

Figure 26: High-level system architecture As visualised in Figure 26, the solution must be integrated at different points in Infront's architecture, and communicate with multiple systems. The web-application will present data collected continuously from the Infronts WebTrader, through their existing solutions for presenting real time market data.

The logging-service will be responsible for receiving logs from the WebTrader, process them into information meaningful for BI-purposes, and appending them to Infront's streaming server. To achieve this, logs has to be ordered and stored in a way making it possible to access them simply and efficiently. A database will be used to for temporarily storing logs. The database will be queried periodically, to continuously produce BI-information appended to Infront's servers.

Appending the BI metrics through Infront's servers will expose them as symbols in the same way as market data. Instead of containing information on a stock's performance, the BI symbols will contain the performance of a BI-metric. An example could be loggins with PID 1 will be represented as a symbol, in the same way as the Apple Inc stock.

The web-application will present the metrics, using Infront's web-toolkit. Infront's web-toolkit will request and display data based on context provided by the web-application. Data will be presented using widgets from the toolkit, including lists, charts and other graphical visualizations. "Infront BI" will serve as a companion to Infront's WebTrader, and therefore

53


should use the same layout and styling as the WebTrader. The application will include user authentication, where data access will be resolved based on the user's credentials.

User characteristics The solution is intended for two types of users; administrators at Infront, and administrators at Infront's providers. Both types will have access to, and make use of, the same functionality. The difference between the two types will be the data available, as providers only will be given access to data related to their clients. As administrators in financial technology, it is expected that the users have working knowledge of how to use web-applications, domain specific concepts, acronyms and abbreviations.

Constraints Stock trading is the subject to strict compliance regulations. Trading must not be tracked or logged in any way, and all market data usage information stored must be anonymized. To facilitate further development of the system, development will be conducted using programming languages and frameworks readily used at Infront when possible. Infront readily uses Microsoft solutions, and the group should aim to use Microsoft solutions if no open-source alternative is found fitting. Although the prototype will only be deployed in a test-environment, the solution should be scalable to operate in an environment with at least similar load as today's production environment. Resources will mainly be limited by the group members other obligations, including other classes and part-time employment. Tasks dependent on resources from the client will be constrained to their availability. The solution must be completed within the time frame of the bachelor project.

Assumptions and dependencies Producing a working BI web-application is highly dependent on successfully developing the components needed to produce data, and on successfully transporting BI-data through systems meant for market data. Development will be started in parallel, with the assumption that this will be succeeded to some extent. Requirements involving dependencies and constraints related to market data technology integration represents solutions with uncertain feasibility. They are listed as requirements with the assumption that they are feasible.

54


Organization of requirements The requirements have been organized in the categories "must", "should" or "could", and are to be prioritized in that order. A result only fulfilling the "must" requirements will be considered as complete, although exploring the "should" requirements are considered crucial to fully meet the client's expectations. The "could" requirements represent requirements outside the main scope of the project, only to be included if all other requirements are filled. The requirements are not prioritized beyond these categories.

Specific requirements

Functional requirements

ID FR1

Title Logging service

Description The solution must include a logging service capable of receiving HTTP POSTs from the WebTrader. The service must be able to identify the type of log received, and store relevant information in a relational database.

Rationale In order to receive and store the data needed to calculate usage metrics

Dependencies Infront WebTrader

ID FR2

Title Logging in Infront's WebTrader

Description The solution must add logging to the WebTrader, logging requests sent to the application controllers. Logs must include an anonymized user identifier to prevent information from being abused if intercepted or leaked; type of log (signifying what exactly has been measured, as singular requests can be used to log multiple events to increase performance); and the timestamp of the log. Logs must be sent to the logging service as HTTP POSTs.

Rationale In order to obtain the data needed to calculate usage metrics

Dependencies FR1

55


ID FR3

Title Logging parser

Description The solution must include a service calculating usage metrics based on the logs collected from Infront's WebTrader. Data has to be formatted as trades. For “counts”, that is to say simple counts of a specific type of event derived from a log: the timestamp should be the end time of the period, price should be the non-distinct count since the start of the day and volume the number of unique entries during the period (count distinct per userID). Data must be forwarded to Infront's streaming servers by appending them as JSON objects to a provided service accepting HTTP POSTs in a given format.

Rationale To calculate and make usage metrics available in Infront's streaming servers.

Dependencies FR1, FR2, Provided service

ID FR4

Title Usage metrics

Description Usage metrics must: - Include login count - Include Heartbeats (user activity) counts - Be grouped by PID - Be calculated continuously for 5 minute periods - Have a meaningful ID and name.

Usage metrics should :

- Include market page access counts - Include symbol page access counts - Be grouped by provider

Usage metrics could:

- Be grouped by Infront (all providers)

Rationale To provide information meaningful for business intelligence

Dependencies FR3

ID FR5

56


Title User login - Front-end

Description The solution must enable authorized users to log in to the BI web-application using the same credentials used to access market data. BI data access must be authorized through Infront's web-toolkit's login interface. Login could be mediated using Infront's single sign-on solution (SSO)

Rationale To enable users to log in to the web application.

Dependencies Infront web-toolkit

ID FR6

Title Home page - Front-end

Description The web application must include a home page, showing "top-level" data related to the logged in user's credentials. This will be data grouped by the provider for provider administrators, and data grouped by Infront (all providers) for infront administrators. Data must be presented using widgets from Infront's web-toolkit.

Rationale To provide a landing page to the web-application, and to make general information easily available.

Dependencies FR5, Infront web-toolkit

ID FR7

Title PID Overview - Front-end

Description The web application must include a PID overview, accessible at all times, showing the PIDs available to the user. The overview should be presented as a table, showing the PIDs, as well as the PID names. Clicking the PID should navigate the user to the relevant PID overview. Data must be presented using widgets from Infront's web-toolkit.

Rationale To provide simple access to PID overviews.


57


ID FR8

Title PID page - Front-end

Description The web application must include a PID view, presenting the data grouped by PID. The view should present some kind of major metrics and rankings, in addition to listing underlying metrics. Data must be presented using widgets from Infront's web-toolkit.

Rationale Display major metrics for all PIDs, as well as making underlying metrics available.

Dependencies FR5, FR7, Infront web-toolkit

ID FR9

Title Symbol page - Front-end

Description The web application must include a "symbol" view, using relevant widgets from the web-toolkit to display the data available for the selected metric.

Rationale To make all metric data available to the user.

Dependencies FR5, FR7, FR8, FR10, Infront web-toolkit

ID FR10

Title Metric search - Front-end

Description The web application must implement the web-toolkits search interface to allow searching for metrics, and accessing symbol views directly.

Rationale To make it easy to find a specific metric, without having to navigate the applications hierarchy.


58


Non-functional requirements

Must

- All web-applications server side code must be coded in C#, preferably using Microsoft's ASP.NET Core framework.

- All client side code must be written in Typescript and HTML.

- The application must be scalable to operate in a production environment with at least similar load as it has today.

- Infront's working language is english. The application, code and documentation must be in english.

- The application must use the same styling and layout as the WebTrader.

- PostgreSQL should be used for databases.

Should

- The front-end web application should be a single page application, built using Angular 2, and bundled using Webpack.

- Build steps should be automated using Grunt.

59


IV.3 Gantt Diagram

60


References 1. Angular (n.d.). Angular Docs (TypeScript), Retrieved May 22, 2017, from

https://angular.io/docs/ts/latest/

2. Jenkov, Jacob (2015). Dependency Injection, Retrieved May 22, 2017, from http://tutorials.jenkov.com/dependency-injection/index.html

3. Onsen UI (n.d.). Onsen UI Documentation, Retrieved May 22, 2017, from https://onsen.io/v2/docs/guide/angular2/

4. Wilmott, Paul (2007). Paul Wilmott Introduces Quantitative Finance (2. edition), Chichester, West Sussex, John Wiley and Sons Ltd.

5. Agile Alliance (n.d.). Agile 101, Retrieved May 18, 2017, from https://www.agilealliance.org/agile101/

6. Agile Alliance (n.d.). Agile Manifesto, Retrieved May 19, 2017, from https://www.agilealliance.org/agile101/the-agile-manifesto/

7. Scrum Alliance (2014). Scrum Guide, Retrieved May 19, 2017, from https://www.scrumalliance.org/why-scrum/scrum-guide

8. Leankit (n.d.). What is Kanban? Retrieved May 19, 2017, from https://leankit.com/learn/kanban/what-is-kanban/

9. Kanbantool (n.d.). Kanban Software Development, Retrieved May 19, 2017, from http://kanbantool.com/kanban-software-development

10. Leankit (n.d.). What is Scrumban? Retrieved May 18, 2017, from https://leankit.com/learn/agile/what-is-scrumban/

11. IETF (1999). Hypertext Transfer Protocol -- HTTP/1.1, Retrieved May 24, 2017, from https://tools.ietf.org/html/rfc2616

12. Webpack (n.d.). Configuration, Retrieved May 24, 2017, from https://webpack.js.org/configuration/

13. Mads Kristensen (2016). ASP.NET Core Template Pack, Retrieved May 24, 2017, from https://marketplace.visualstudio.com/items?itemName=MadsKristensen.ASPNETCoreTemplatePack

61

https://angular.io/docs/ts/latest/

http://kanbantool.com/kanban-software-development

https://tools.ietf.org/html/rfc2616

https://marketplace.visualstudio.com/items?itemName=MadsKristensen.ASPNETCoreTemplatePack

https://marketplace.visualstudio.com/items?itemName=MadsKristensen.ASPNETCoreTemplatePack

https://www.agilealliance.org/agile101/the-agile-manifesto/

https://webpack.js.org/configuration/

https://onsen.io/v2/docs/guide/angular2/

https://leankit.com/learn/agile/what-is-scrumban/

https://www.agilealliance.org/agile101/

https://leankit.com/learn/kanban/what-is-kanban/

http://tutorials.jenkov.com/dependency-injection/index.html

https://www.scrumalliance.org/why-scrum/scrum-guide

B A C H E L O R P R O S J E K T - student.cs.hioa.nostudent.cs.hioa.no › bachelorprosjekter › data › 2017 › 39 › docs › Fina… · II.1.1.2 Kanban 10 II.1.1.3 Scrumban

Documents