Home Analysis and system design Bummer, or how a favorite IT project failed

Bummer, or how a favorite IT project failed

by admin

"His example to others is science."

Preface

This is a sad story about the failure of a project that I thought was potentially 100 percent successful. And why it ended up being a bust, I still don’t really understand.
In my life as an IT guy, I’ve been involved in the development of many successful projects. Most of them were relatively simple in idea and mediocre in implementation.
To be fair, I will mention an exception, a remarkable project in my opinion. It was quite extraordinary in concept (the idea was from a high-ranking bank economist) and good in implementation. We implemented it in assembler. The database was a set of files for direct and sequential access. The project was successfully maintained.
Once, while participating in another mediocre development of a bank’s operating day, I was faced with the need to calculate some indicators whose values were regulated by a national bank. This was already beyond the scope of accounting debits and credits. And I had read the books "Economics" (by Campbell and others) and "Money" (by Dolan), and I started to read "Microeconomics" (I don’t remember the author, but it was foreign). All this gave birth to the idea of analytics, as a kind of analogue of medical analytics. All indicators, which reflect the state of the business, should be defined. This means that precise rules for calculating the indicators should be defined, and a mechanism for their calculation and use should be implemented. Ideally we would see workers with screens showing the current and predicted state of affairs, for which the employee is responsible. And the worker asks questions like :

  • And why such a jump in such an indicator?
  • And who made such-and-such an operation?
  • What happens to the state if you perform such-and-such an operation?
  • What happens to the state if you change such-and-such indicators to such-and-such values?

Etc.
And the employee makes decisions based on all this. Would anyone turn down such a prospect? After all, this is a real modern workplace for the creative economist and manager.
In the end, the project came to fruition. I naively thought that it would be a success. After all, the time of directive control is over, and it means that we need a mechanism to keep a hand on the pulse of business (I even thought to give the project name "Business Pulse"). And what happened, we’ll see.
I fell in love with the project and was quietly confident in its success. But, alas … I hope that, at least for beginning developers, the story will be useful.

Project history

"A fairy tale is soon told, but it is not soon done".
All names of firms and banks are not real.

Company M. Home

At M, I was involved in the bank’s currency operating day project(ODB). And that’s where I encountered the beginnings of analytics. Regulatory documents came in that defined a variety of regulations for commercial bank activities and the regulations for their application. The regulations defined the risk framework for the activities of the bank. And then, inspired by the translated books on economics (God forbid, I read the Soviet books on economics), the idea of Analytics emerged, a project that allows the user to define and calculate any economic indicators and monitor them for compliance with normative or other restrictions. I talked to the executives (and they were three ladies: president, chairman and director!!!) but I could not convince them of the necessity of an independent project. So I actually found myself out of a job, bored and sad. But then I was offered to work on the side – to develop a project of multicurrency bank operating day (ODB) for the NP firm. I began to work and eventually joined NP on a permanent basis to develop a project for a multi-currency ODB. But the idea of analytics remained in my heart.

NP firm. A modest continuation

So, in NP I am developing a multi-currency ODB. Keeping in mind the norms mentioned in the previous paragraph, I implemented them and hooked them up to a reference that gave the definition of the index, the formula for its calculation, and the current value. All this was implemented in a straightforward way, without any unification and deep system. Another banking product exhibition is taking place in Moscow. And we are exhibiting. A banker comes up to us and starts to get acquainted with our product. Soon she asks: "And how are you doing with the standards? I show her a table of current values of standards, she presses F1, and then comes the definition of a standard and the formula for its calculation. She was immensely pleasantly surprised, saying that she had never seen anything like that. And here I was strengthened in the idea of analytics. If a head-on solution for a few metrics pleasantly surprised the client, what would happen if she was given real advanced business analytics. All in all, I was even more enthusiastic.

BSB Bank. Personal systematic commencement

I once demonstrated my currency transaction day project at a BSB bank.
The automation at BSB was handled by my former boss from the PSB bank. He looked at the project, had a couple of coffees with me, and invited me to work for him. And we were already implementing the project in one of the banks. And I, alas, was not very eager to implement. So I agreed to make the switch. And here I am in the BSB. I coped with the work quickly, and I had a lot of free time. I began to learn Delphi.
Got acquainted with the exploited tasks in the bank. It turned out to be a whole conglomerate of different types of tasks. Each one is autonomous and has its own approach. Once again I realized that I need a unified banking analytics. I worked on it on my own initiative. I sketched out the concept and beginnings of a technical project, and I refer the idea to my boss. But just when the boss recruited personnel from the PSB computing center, and they show some rudiments of banking analytics, which they used in the PSB. All implemented on Oracle Discovery. And the chief himself comes from PSB, and the IT people are from PSB, and they all know each other. So the boss decided to use what the guys from PSB would bring and not take any chances with new approaches. So, I got screwed. But I continued to develop the project on my own. There was enough time. I educate myself – I read thick books: "Financial Management in commercial banks" (author Sinky), "Bank Management" (the author of Rose), "Financial Management Company" (author Smith, etc.). And I’m getting even more solidified in my idea.
When I left BSB, I found out that BSB was thinking about buying SAP, which they eventually did. Then they implemented it for several years and I don’t know if they did. Yes, it was fashionable to turn to Western products. The explanation is simple: it meant systematic foreign trips at the expense of the bank.
I continue to be itchy for analytics. So I decided to promote it with a private lending example. Briefly about the essence of the problem. There is a portfolio of credit applications. In the general case the entire portfolio can not be satisfied: there may not be enough resources. Plus there is a set of norms which should satisfy the bank. So there are problems of choice. And where there are several variants of choice, there is a problem of optimal choice.
The optimization criterion may be different, e.g. :

  • Integral interest profit for some period
  • Net present value of choice for the same period

The case is complicated by the fact that each client is characterized by some risk. So the criterion must be weighted by the probability of repayment of the loan. I managed to reduce the problem to a linear programming problem. The client risk was set by the model of empirical risk, which I had to develop myself. The difficulty was the normative constraints, which were set by non-linear formulas. It turned out that they were non-linear only in their form and could be reduced to linear ones by simple identical transformations. The risk could also be set a priori.
So, the staging is done. I need to get the bank to approve the project. I go to the management. Management can not evaluate the proposed ideas, and I am sent to some semi-scientific department (I forget the exact name), and I get into a consultation with a candidate of economic sciences, a former Narhozov (Institute of National Economy). I do not know why, but he immediately turned against and started saying phrases like "i, j all know how to write. In the subtext can be heard: "Come here is not important" and "… Judge your friend is not higher than a boot. And there is no way to explain him that the project does not claim to be a science, and offers a specific development and an algorithm for it, so you can start programming. But there is no way to push this into him. In the end, I did not get any recommendations. "So what the hell. Let it be as it is." This was the second time I encountered industry science, and the second time I got on its horns.

The MRC company. Personal systematic continuation

The chief is leaving to head the MRC. He’s taking me with him. The firm is cash strapped and can fund multiple projects. I start probing the development of a business intelligence project. I continue to sketch out the technical project. I develop the database structure using CASE tools. Once again I have had the opportunity to take the initiative. RTGS (Real Time Gross Settlement) system is in full swing – the real-time realization of large payments. The project is supervised by Europe. Well-groomed European consultants come regularly, and we treat them to instant coffee from unholy rusty tins. Our management travels to Europe on a regular basis. The management is serviced by interpreters from my department. And I have a bookcase full of valuable translations on banking issues. I cannot understand most of them, either because of the complexity of the subject or the quality of translation. All right, RTGS is being developed. But what about small payments? They are supposed to be implemented in clearing mode. There is no definition or intelligible terminology for clearing. And so I take it upon myself to develop a clearing project. I develop a glossary of terms and start a technical project. I give the dictionary to the general public for review. My enthusiasm just keeps pouring out of me.
But then new political times come. Vinnikova, the chairman of the National Bank, is arrested. And my boss was put in charge of the MRC at Vinnikova’s suggestion. And so the squeezing out of everyone who was directly or indirectly related to Vinnikova began. The chief gets fired. And I begin to be pressed. Even exiled to a small room far from my office. A new boss comes in. I bring him my clearing project for review. The boss keeps the project for about a month. I waited and waited, and then I turned to him. And in response I heard: "The development of the project is not part of your direct job description". So much for the initiative. I ask for the project to be returned. Chief crumples, crumples, but still returns the project to me and the project is still gathering dust on my balcony. By the way, I wrote an article about clearing for an industry magazine. It proposed a model which would determine the time lag of clearing, optimizing the use of clearing resources. The article was accepted and printed without any obstacles. Encouraged, I developed the idea further and submitted a second article to the journal. And then I realized from the behavior of the editor that the channel was blocked. And it became clear to me that it was time to leave on my own, without waiting to be asked. I contacted the firm SK and offered an analyst project. Immediately I get an invitation to work. So I’m on the right track.

Company SC. The beginning of venture capital financing

So, I’m in SC and the head of the Zero project. It’s a bank analytics project. There’s no one besides the manager yet. By myself for now. I start the implementation in Delphi. Three months pass and the technical director calls me and asks me to introduce him to the state of the project. I do. I already have something. And the director says: "Why not switch to self-sufficiency – sell the project in parts that have already been implemented? I’m taken aback. I began to object that MAZ does not sell wheels, but sells dump trucks. They bicker and go away.
I was beginning to feel the scrutiny of the chief engineer. He was my immediate supervisor. The whole design ideology came from him. A man of talent with both a tough management outlook and a soft carrot of praise. He probed my business all the time and once explained himself. He had developed an interpreter for a subset of Pascal and all the bank reports of the firm’s main product were done on scripts from that interpreter. The scripts were called OPR, I don’t remember why. So, he said the project must be done in ODA and all analytics should be reduced to reports written in ODA. This was not yet an order (and of course he had the right to order it). We started a discussion. I cited the object approach and a much more systematic approach to indicators down to their coding and the approach to analytics not as reports, but as modeling of economic situations and analysis of the consequences of modeling as the pluses of my approach. All in all, there was a nice normal discussion. Such discussions continued several times. That’s good, that’s right. However, neither he nor I change our point of view. Then the boss suggests that we have a public debate, in which each side presents its own list of arguments. Fine. But on the day of the discussion, I find out that the boss has corrected my list to his own taste from his boss’s perspective and presented the corrected list to the public. This shocked me. After all, you as the boss could order me to work as you see fit and I, either have to accept it or quit. But if it’s a free discussion, don’t shut me up. We’re on an equal footing. Anyway, I lost the argument, but not by a landslide. "The calf fought the oak tree." I got a good lesson in corporate ethics and started looking for a new firm. And I found one – ATL.

ATL Company. The end of venture funding

And then, finally, I get carte blanche for all phases of development. And permission to recruit a team. But while I was still working alone, the boss had already invited potential clients to view the product. And now a banker from Kazakhstan wants to buy the project. Good thing the boss asked me if I was ready. I refused to implement it: I had to develop and implement it myself – was not too much. So I started to put together a team. There were never more than 8 people in the team. Each person was given a decent independent piece of the project. I drew up the terms of reference for it, and then I went ahead. After a month or two it was obvious whether the person was suitable.
So, the project began to develop at a rapid pace. Some features were prompted by potential clients. For example, I am demonstrating the project to the French. I got to the charts of dynamic series. And then one Frenchman asks: "Can you tell me why there is such a jump in profits here? And then the thought occurred to me: "How could I, an idiot, not have thought to ask myself such a question?" This led to the emergence of operational decomposition – a set of all the operations that change a given indicator at a given time. And from the operation you can get to its executor.
Storage and processing of dynamic series, scenario modeling, virtual indicators, statistical testing of hypotheses, regressions, search for the best regressions, processing of a portfolio of financial instruments, all sorts of decomposition of indicator values, all sorts of comparators – everything was implemented quickly, in my opinion. The project was getting more and more solid. I tried for the sake of interest to implement Web-interfaces to create Web-services. This turned out to be uncomplicated, but further web-thematic we did not develop.
We began to exhibit at IT-exhibitions. I demonstrated not a presentation, but the real work of the product. Later I gave that up, as it was too much stress. Even my hands began to shake when I had to actually demonstrate the desired feature. There was a lot of interest in the product. IT executives from a reputable bank came. The next day a whole brigade from the IT department came. Then came a brigade of bankers. And I show them everything. Well, I think the deal is done. Alas. But most of them came from IT. And they were so meticulous with their questions that one day I asked them directly: "So do you intend to take it or are you just curious?" And the answer was: "No, we’re not going to take it, we’re just doing something similar, so we wonder how others are doing. I asked to see their project. They didn’t refuse me. The project was implemented in java. It dealt only with the generation of indicators. And this generation took too much time in my opinion. And, therefore, the generation was going on at night.
One beautiful spring day I was called to work. I have to show a project to an important banker from St. Petersburg. The banker really liked the project and asked me to come to Peter and show it to the public. We go. The entire bank management team is present. We were introduced and then, before the presentation, heated discussions among the managers began. One party was for the project there, and the other was for us. We listened and listened and asked to interrupt the debate. Like, we’ll show the project, and then debate. And so we did. In the end, we decided that we would transfer the whole database of our project to Progress in St. Petersburg, keeping all the features we already have in it and implementing some new features in addition to that. Then we will show all this and then decide the fate of the project. We have implemented what is required and are going on a business trip to St. Petersburg. And now the time has come to go. And then the boss says no. What and why, I never knew, since I was not privy to the business intrigues. And Peter was screwed.
There were visitors from Paris, too. The head of the delegation was some descendant of Russian immigrants.
He knew Russian quite well. Apparently, he liked the project and as a sign of respect(?) he gave me a laptop from IBM, though not quite the latest model. The laptop still works. Soon the boss ordered me to prepare a project for Paris. Wow! We translated all the documentation into French. We localized the project. We’re exchanging data with the bank in Paris. We’re getting ready to go on a business trip. And then the boss says our patron is being promoted to another firm, unrelated to our subject… And everything goes quiet.
My former MRC boss was the head of the strategic research department at the PSB bank. And he let me put the project to the test. And what do I tell you? – Never try to exploit a customer’s raw project. Only damage the reputation. And so it happened. After a couple of failures – coolness from the bankers. Yes, and the view of all the type of "Who needs extra work without extra pay. In the end, my acquaintance at another meeting over a drink and discussing the current political moment uttered the sacramental phrase "With this policy no need for analytics.

Promissory Notes

And then the opportunity presented itself to implement a small subset of analytics on the example of promissory note processing. The work with the subjects was completely taken from the analytics. Well, a few indicators came in handy as well. Quickly implemented Bills of Exchange project. Project is implemented. Support is going on. And then came the thunder. The control authorities received a complaint that the project was laundering money, and the project does not do what is necessary (it seems it was because we worked with the son of the customer of the project Beksel). And now we developers have to demonstrate the project to the auditor. Good thing we set out the TOR in detail and clearly implemented all the points of the TOR in the project. Demonstrated to the auditor the work of the project on all points of the TOR. There was nothing to complain about. The project cost very little money, and I can’t imagine how we could have made any profit out of it. At least we did not cash in on it.
The project was saved and we were already thinking about developing it. But then the president banned the bill form of settlement. Although we were asked to preserve the entire infrastructure of the project (work with business entities…), but there was no question of money and we had no reason to work.

Criminal Search

And then another project came up, which could and should be connected with analytics. An interesting potential customer showed up. He describes the subject of automation in the following way. There are many subjects and many relations between them. Direct and derived attributes characterizing the degree of financial criminality of the subject should be defined and on the basis of criminality attributes the suspicious subjects should be detected and tracked. The prospective customer said only that it was necessary to keep a file and a file on each client and to collect information about all the relationships of the subject. The main data providers are: the Ministry of Internal Affairs, banks, tax authorities, the Social Security Department, Customs, Interpol…
The debate began about whether to develop it ourselves or take a purchased product. The customer was more inclined to the purchased version. It was quite understandable: foreign business trips, appealing advertising of the products (the products themselves had no demos and could not really be used), and the money was from the budget, so why count it. I began to develop the TOR. I soon realized that it was unlikely to be finalized. The thing is, my new boss was trying to get all the documents, structures, and files to be defined at the ToR stage. In fact he wanted to squeeze the technical project into the ToR.
In addition, each decision had to be minuted for the signature of the customer and the developer. Therefore, meetings with the customer were held on each point of the TOR. I expressed my doubts about the success of such an approach, and began to develop an independent project on my own initiative, using Delphi. I was just curious. The main information structure – the oriented loaded graph of relations. Nodes of the graph are subjects, edges of the graph are relations, loaded with type, size, date of the beginning of relations, date of activation of relations… There can be different searches in the graph: chains of homogeneous relations, chains of heterogeneous relations, chains of the given length, loops, chains of relations, with sizes exceeding the given limit, etc. etc. I developed the corresponding classes and implemented the project relatively quickly. I demonstrated it to the head of the supposed criminal search project. On several tens of thousands of subjects, hundreds of thousands of relations, a dozen and a half types of relations, any chain was found in about a second. And literally the next day the supervisor said that the customer had terminated his relationship with us. As you might expect, the matter went no further than the ToR. It took us about a year. The moral: you shouldn’t make the client too uptight. And maybe the boss didn’t even want this project. The thing is that he was also responsible for the ERP project for the KGB, and the project was on fire…
And I, working with relationships, got interested in the Prolog language, or rather Visual Prolog. I started trying it out. And surprisingly, in Delphi, chain search implementation required a lot of effort and program size, but in Prolog I did it with a compact declaration. That’s what the problem-oriented and declarative nature of the language means. The description of the problem domain is declarative and therefore it was relatively easy to fit into Prolog declarations.
But I didn’t really like Visual Prolog’s form rendering capabilities. And the idea dawned on me that every specific problem needed its own language. Report design and report generation language, data input design language, data output design language, communication language, relationship processing language, list processing language, data description language, business intelligence language, service placement and management language, accounting language, … In short, we need subject-oriented languages, not universal ones. It’s just a matter of standardizing the interfaces between the resulting modules. Why do I need all power of C# with Visual Studio, if I, for example, develop only forms, or only reports, or only exchange with the database…
The project has been included in analytics in the Relationship subsystem.

A digression about technical writers

When I started working with the above manager (he is also a former physicist), he began to complain that none of his young subordinates are not able to properly write documentation. I fully agreed with him and all documentation on my projects I wrote myself. Right down to Powerpoint presentations.
Now it seems to me that the problem of technical writing has become even more acute. People have begun to read little and write only Internet comments that are sometimes embarrassing to read.
What is the most important language? – Your mother tongue. You should know it first.

Integration attempt

The project is interested, but the project is not being bought. I got worried and started looking for reasons. And at the time, the main banking IT product on the market was the bank operation day (ODB). So I decided to expand the project and make an integrated project that included: ODB, HR, deposits, loans, in-kind accounting, operational accounting. I didn’t have to talk my boss into it. Things went well with the UDB. But the loans and deposits were an embarrassment. The point is that I did not have to recruit the staff for these projects myself, and had to limit myself to people from one promising, but unsuccessful project. The guys are young, perky, apt for system programming. Sockets, TCP/IP, Internet, patterns… A month goes by, but no progress. But every morning the guys vigorously discuss "Formula 1". So you hear "And Schumacher, and Alonso…". After discussing Formula 1 switched to the algorithm of division of Internet traffic between employees. Yes, there was a socialist relic of a fair division of traffic between employees. The thing ended with me suggesting that the depositors and loan officers quit. Which it did.
And then new times began. The firm was owned and run by a nice businessman. A very religious man. And there was a thick layer of his fellow believers in the firm. And everything was going more or less smoothly. And then suddenly, I do not know why, a new manager appears – a freshly minted KGB retiree of high rank. Soon we received an order to design an ERP for the KGB. As time went on, the project began to stall and the ERP development team had to be urgently strengthened. My project was a venture project, and not bad guys were working there. So they began to pull people out of my project. I couldn’t say anything against them: we were sitting on the necks of other projects. So I was soon left alone again. And the KGB layer in the company kept growing. A project to develop encryption-decryption equipment was started.

Final

In the end, I was left alone on the Analytics project. They threw me an order for Ministry of Internal Affairs to finalize ASP-project. Finished it. And then it turned out that the firm was going bankrupt. It seemed to be a clever business move: along with bankruptcy process there was a new firm with people from our bankrupting firm, all hooked up with KGB – development of cryptographic hardware. And the old firm went bankrupt. And my analytics became a pet project. But by inertia, my interest to the project continued for some time. Then came the time for .Net. I realized, that it was already relatively easy to move to SOA-architecture. I transferred almost all of the business logic from Delphi to C#. Got to know WCF. Divided the services. Defined the interfaces. Implemented the service-client gateways. And there it was SOA-architecture. A thought was to implement a simple service orchestration, but it seems that this is no longer necessary. This feature has become a system feature.
The phone rings one day. The call came from IT people, who wanted to develop business analytics. They approached the owner of the tut.by portal, for whom I had once developed a currency UDB project. My former boss (God rest his soul) always came up to me at exhibitions and I told him about my successes. He made some good marketing points that got me thinking. So he referred the guys to me. They told me about their plans. I told them that their ideas had long been implemented, but had been shelved. So I showed them the project. They understood that this was a very labor-intensive business and offered to be my project’s pusher to the banks. And I had already entered the stage of pessimism about the prospects of pushing into banks. But the guys begged me to try.I told them to go for it. I provided them with promotional materials, a letter to the banks, and they went to push. The result was predictable.
I still can’t determine the reasons for the failure.
I tried to give the project to EPAM. They reasonably answered that they develop only to order, and they don’t deal with venture developments. There is a High Tech Park in Minsk, HTP, which operates under the auspices of the President of the Republic. There is a business incubator in it. I sent there a proposal to give them my project for free. There was no response. I looked up incubators on the Internet and found this. "An incubator (from Latin incubare, here to hatch chicks) is an apparatus for artificially raising young farm birds from eggs. It produces fat, but unfit for non-incubator, real life individuals. Sounds about right.

Project description

"Not all gold that glitters"
And now more details about the long-suffering project. This is to assess whether my expectations were reasonable and worth worrying about. This is not an advertisement for the project. The project is dead. I am willing to give it away for free in good hands. If anyone is interested in some of the details of the project see The old original description(so perhaps more interesting) is here

Principles

  • SOA
    A service is defined as a self-contained, collaborative application whose functions are provided through a developer-announced API. A service can be hosted on any computer on the network. In the client-server architecture, there was only one service: the DBMS. In SOA, there can be as many services as you want, and they can be located on different computers on the network.
  • WCF client-service and service-service communication.
  • DBMS
    There are no stored business procedures in the database. The DBMS is fairly loaded with standard save, update, and retrieve queries. The absence of stored business procedures makes it much easier to change the DBMS. Especially if you still try to limit yourself to the SQL standard.
    And if it turns out that the database server is underutilized, it is possible to place a service to work with the indicators, for example.
  • Reports
    Reports should not have any business functions. A report is just a mapping of a group of indicators to a given form. And the indicators themselves should be calculated in the business layer by the indicator machine service.
  • Interfaces
    Any reference to the service must be through interfaces only.
  • Groups
    The same-type objects (indicators, subjects, operations…) can be combined by any admissible logical criterion. There can be as many groups as you like.
  • Explorers
    Hierarchical structures must be displayed in a single explorer

Basic Functions

  • Maintaining subjects and their relationships. Search by relationship. Maintaining groups of subjects.
  • Maintaining indicators and their time series. Maintaining groups of indicators. Values of the indicators can be numerical, text, logical, calendar. Rule for calculating the indicators – any composition of standard mathematical functions and aggregate functions. Examples of aggregate functions : maximum by subjects, average by time, minimum by time…. The values of the indicator are presented in several dimensions of reality : fact, plan, individual forecast, statistical forecast, scenario forecast. Surprisingly, nowhere in the literature have I found a systematic list of business indicators with their definitions, codes, names and calculation rules.
  • Statistical analysis of dynamic series. Testing statistical hypotheses. Finding statistical parameters.
  • Regression analysis. Finding the best regressions.
  • Portfolio maintenance. Obtaining a payment calendar.
  • Forecasting. Forecasting with linear trends. Forecasting by financial instruments portfolio. Forecasting with statistical deviation of forecasts from reality.
  • Scenario modeling. Different types of scenarios : operational, portfolio, indicator.
  • Analysis. Various kinds of analyzers (decomposers): decomposition by indicators, by subjects, by currency, by performers, by operations.
  • Comparisons. Various kinds of comparators : comparison by time, by subjects, by indicators.
  • Real time display. Each user has his own type of scoreboard, which displays a group of indicators.
  • Maintaining a chart of accounts, a hierarchy of accounting objects, a hierarchy of business transactions and their mapping to postings.
  • Elements of budgeting : the resource allocation game.

Basic Technologies

  • ETL. This is a non-unified data pumping module from primary information source subsystems. The primary source is accounting.
  • Explorer as a universal display of hierarchical structures. Hierarchies of indicators, subjects, reports, queries, accounting objects, and operations are displayed in a unified way.
  • Groups as sets of objects united by some common external semantics. For example, a report is a group of indicators displayed according to a given form.
  • Events. The event monitor monitors the birth of events. An event is specified by a logical function. Event types : internal business event, external business event, calendar event. Monitor is common for all services. Each address space has its own event notifier. It is notified by the event monitor. The application subscribes to the events.
  • Dynamic Forms. Users can design screen forms of the following types: list forms, matrix (=table) forms, hierarchical forms.
  • Elements of object circulation.
  • Services.

Functional services

  • Business Core. Provides a business object to the client for use and takes the object away from the client upon request. Caches objects. Manages the communication between the business tier and the storage tier.
  • The current state of the subject. This is the set of subject’s indicators as of the current date.
  • Simulation state. This is a set of forecast indicators of the subject at a given date under a given active development scenario.
  • Statistics. Allows you to find statistical dependencies.
  • Indicator Analysis. Statistical and graphical analysis of real and virtual indicators.
  • Subjects. Maintaining subjects and navigating through their set.
  • Relationships. Allows you to analyze any complex chain of relationships.
  • Accounting entries. Prompts accounting entries for a given business operation.
  • Real-Time Business Scoreboard. The current values of the indicators are displayed at a set pace. The display is tabular and graphical.
  • Portfolio of financial instruments. Maintaining financial instruments, navigating through their multitude, working out passive evolution scenarios, defining the payment calendar.
  • Budgeting, point or spread over time.

Technology services

  • Identification and authentication. Identifiers are generated for new business objects. When a user logs in to the application, their password in the application is checked.
  • Access. Two types of access are implemented : by permission, and by denial. Levels of access : type, method of type, instance.
  • Indicator Machine. Calculates the values of derived indicators and stores these values in the database. The indicators are loaded from the database.
  • Event Monitor. A user-generated set of events is monitored. An event is reported to a siren, which is located in the address space of the application
  • Notifier. The application is notified of an event that has occurred. The notifier generates an identical event. The client subscribes to the event itself or the technologist subscribes to it.
  • Object Conveyor. Business objects run through the process pipeline.
  • OLAP. Feature : The basis for OLAP cubes is not so much the raw data of OLTP systems, but rather a layer of calculated indicators over OLTP systems. This makes OLAP-development simple and more productive than directly on operational data. The dimensionality of the measurement space can be increased by hooking the appropriate subject to the indicator and making its characteristics the dimensions of the cube.

Objects

All objects are divided into :

  • D-objects. This is bare data with no business functions.
  • B-objects. These are business objects containing D-objects and business functions.
  • P-objects. These are view objects that contain D-objects with their mapping attributes.
  • C-objects. These are communication objects that serve to transfer D-objects between different address spaces.
  • DB objects. These are objects for mapping D-objects to the database and retrieving D-objects from the database.

Object Interaction :
Bummer, or how a favorite IT project failed

D-object – data object

Designed to store data. A D-object is a B-object without business functions. A data object contains only data and no business functions. Name : <name> D. Example : HomoD.
All data objects form a D-tree. The root is TopD.
D-objects contain non-business methods :

  • CopyTo
  • CopyFrom
  • Clone
  • GenId
  • GenParentId
  • ToString

About Identification

TypeId is the identifier of a type. In the Tip table, each type corresponds to an entry with Id=TypeId. There can be any number of instances within a type. They are distinguished by the Code attribute.
Code is the identifier of the instance within the type. So the TypeId and Code pair gives the instance identifier. However, different data providers can supply identical instances. For example, there can be as many instances as you want. To identify to this level, let’s use an instance qualifier IQ, which is unique within all instances for the lifetime of the solution. Then the triple of TypeId, Code, IQ is the identifier in the Tip field. You could do with a single IQ, but it’s inconvenient for manual work with the database.

B-object – business object

These are all entities that are elements of an application domain, which is represented as an interaction of business objects. Designed to store business functions. Example of business objects : person, account, payment. Example of non-business objects: all widgets(=controls).
A business object contains a data object and a business function. Name : <name> . Example : Homo
Canonical correspondence : <name> D <–> <name> . Each B-object has methods :

  • Save
  • Retrieve
  • Delete

All business objects form a hierarchy. The root is TopB

C-object – communication object

Designed for data transfer. It contains data in a format suitable for data transfer between different address spaces. Name : <name> C. Example : HomoC
Canonical correspondence : <name> C <–> <name> D.
All communication objects form a hierarchy. The root is TopC.
C-object methods :

  • CopyToD. Copy itself into a D-object
  • CopyFromD. Copy to itself from D-object

P-object – the represented object

Designed to display data on visual forms. Contains the data object and visual display parameters.
Name: <object type> P. Example : HomoP
All objects in the view form a hierarchy. The root is TopP. Canonical correspondence : <name> D <–> <name> P. Many P-objects are the same as D-objects because they do not need rendering parameters.

F-object shape

Serves for the visual representation of data objects.
Name : <object type> F. Example : HomoF
All forms form a hierarchy. The root is TopF. Canonical correspondence : <name> F <–> <name> P.
All functional forms are inherited from TopF. Basic properties and methods :
Each form has methods :

  • Get object
  • Display the object on form fields
  • Generate object from form data
  • Save the object
  • Delete object

Each form has its own execution context – UserContext, which is written in TopF.
All grid tables on forms can be output in Excel. This method is implemented in TopF. Any index code or valid calculation rule can be entered in the Excel column.
Processing path : input attributes on form –> D-object instance –> B-object instance –> business function –> D-object instance –> DB-object instance –> table tuples in database.
Display path : table tuples in the database –> DB object instance –> D-object instance –> P-object instance –> object attributes displayed on form.
The uniformity of form behavior is ensured by a common form root. An alternative method is to use interfaces. It is sufficient to implement an interface in a component and for the calling program to check for it. This implementation prevents overloading of the root form.

DB-object – database object

Designed to communicate with the database: save the data object to the database and retrieve the data object from the database.
Name : <name> DB. Example : HomoDB
Canonical correspondence : <name> .D<-> <name> .DB.
All DB objects form a hierarchy. The root is TopDB.
All business objects are registered in the Tip table with a unique Id. Object specifics are mapped to specific tables.

Architectural Diagram

Bummer, or how a favorite IT project failed

Data Level

It deals only with D-objects. They form a D-tree. The top of the D-tree is the TopD object. The purpose of the TopD object is to ensure that all D-objects behave the same way.

Level of stored data

It deals only with D-objects and DB-objects. DB objects form a DB tree.
Functions :

  • Extract data objects from external memory
  • Saving data objects in external memory
  • Caching of data objects

By means of a DB object, a D-Object can be mapped to multiple tuples and multiple database tables.
There is no business logic in the database. The DBMS takes care of all the SQL queries. If you want, you can put the business service on the DBMS server. You can put everything on one computer if you want.
ADO.Net is used to access the database. The layer deals with two data streams :
From the database to the logic layer. The data layer extracts the right data from the database and forms business data from it, which at the business logic layer will enter the business object.
From the business layer to the database. The data layer gets business data from the business logic layer and puts it in the database.
There is an option to cache
nd data caching. Caching is saving an object in RAM if the object is frequently used. So in bank settlements you need a correspondent account all the time. By caching it, you won’t need to access the database often – it will be taken from the cache.
DB object has functions :

  • Retrieve data from a business object
  • Transfer data to a business object
  • Save data in the database
  • Extract data from the database
  • Update data in the database
  • Delete data from the database

Properly handle multi-object transaction (do not do Commit and RollBack by yourself)
One of the parameters for a DB-object call is DB-connection
Each object type has a table in the database with the name of the object type. All objects have unique Idand are represented in the table Tip. This table has attributes that are common to all objects :

  • Id
  • PId
  • TypeId
  • Code
  • Name
  • CreatedOn
  • Description
  • StateCode
  • Author

All objects in this table form a hierarchy, which is a type hierarchy. This means that each type is represented by a record, and all instances of a type are subordinate to that record (PId exemplar = Id type).
For a type its Id is TypeId; Code is the identifier within the type.
Object details are represented in tables corresponding to the type. For example, the attributes of the Homo object are represented in the Homo table.
All attributes of a type are contained in the mapping of the Tip table and the table corresponding to the type.
The corresponding representation is referred to as <type> _v. For example, Homo_V.
The top of the DB-tree is the TopDB object. The purpose of the TopDB object is to provide the same behavior for all DB objects.

Business level

It deals only with B-objects. B-objects form the B-tree.
Functions :

  • Implementation of business functions
  • Provision of business interfaces
  • Provision of business services(collaborative application):
  • WCF services
  • Web Services

The top of the B-tree is TopB. The purpose of the TopB object is to ensure that all B-objects behave the same way.

Presentation level

It only deals with P-objects and F-objects.
Functions :

  • Requesting data to the business level and receiving data objects from it
  • Displaying data in forms and reports
  • Forming data objects and transferring them to the business level

The form visually displays an object instance at runtime. Calling and completing each function of a custom object causes that event to be marked in the .Log. All business logic must reside in business object methods. The form should call these methods, but the form itself should not calculate, for example, interest on a loan, maturity, etc. There should not be any business logic on the form, only user interface logic. The user interface logic includes :

  • Managing the visibility of interface elements
  • Managing groups of related buttons
  • Font type
  • Color, etc.

F-objects form an F-tree. The top is TopF.
The purpose of the TopF object is to ensure that all F objects behave the same way.

Mandatory principles

Entropy Minimization

Every developed project is subject to some kind of thermodynamics: every project has entropy. An increase in entropy is a tendency to destroy the structure (=order), a tendency to disorder. Striving for minimal entropy requires uniformity. "Even if it is ugly, it is uniform. All variety must be conscious and pursue a clear functional purpose, not aesthetics, originality, etc. No unreasonable diversity is unnecessary entropy. Fighting entropy is the most important thing. Some programmers already have one module containing a lot of entropy. Examples of unwarranted diversity :

  • Different names of the same entity in different modules
  • Different code layout discipline
  • Different drawings for buttons of the same purpose
  • Different tips for buttons of the same purpose
  • Different programming style. So sometimes the list loop starts with 0 to Count-1, and elsewhere it starts with 1 to Count.
  • Different naming style
  • Different comment style – sometimes before the code {will now be presented…}, and sometimes after the code {this was described…}

Priority of global goals over local ones

The local level refers to the module level rather than the application level or the application level rather than the group of homogeneous applications. So striving for local optimization can lead to global non-optimality, striving for local elegance to global clumsiness. Example 1: the desire to get away from global variables can make the system heavier by requiring the recreation of an object that could have been passed through a global variable. Example 2: when creating a new form, it may seem like it doesn’t need to be inherited from a base form. At first, everything is O’k and resources are saved. But then the form may be required to perform a common form function that is already implemented at the level of the common form, but is not present in the private form. So the form will need to be finalized, duplicating what is already done in the common form.
Adequacy (non-redundancy, but completeness) of the representation(Principle of minimal redundancy).
Any display must be configured for context – take into account all the parameters already set. So if the client is a natural person, the explorer should display only accounts of natural persons; If we are talking about loans, the accounts explorer should display only loan accounts. This applies not only to ouput interface elements, but also to input elements: comboboxes, sheetboxes – data providers.

Covariance when changing the underlying subject

Many Query now contain initial parameter values tied to the current base entity. This is convenient to view in DesignTime, but it can lead to project intolerance. To prevent this from happening, you should always set the value of the query parameter in RunTime.

No visible debris

Garbage is what is not used. Unused interface elements should be invisible.

Layering

Display functions should not be in a business object.
A business class implementation should not use view-level objects and methods.
Database communication functions should not be in a business object.
The business class implementation should not use storage layer objects and methods.

Weak external cohesion

The condition of weak cohesion of a class (external cohesion) is a measure of the use of other classes of the same subject area in the class. It is desirable to have loosely connected classes. This property is called insularity.An object should not use as a parameter in its methods the object of a class located lower in the tree or using the object as an aggregation element. Let B be the descendant of A and A has method A.Do(prA:B). Then it must be removed from A to B: B.Do(prA:A).

Strong internal cohesion

The condition of strong class cohesion (internal cohesion) – the class must contain applied methods, united by semantics. So the presence of mapping methods in a business object violates this principle. Side by side – it shouldn’t be. A class is not a featureless conglomerate, but a monolithic entity subject to a common business purpose. What can fall apart must be disconnected. Weak internal cohesion should cause the business object to disintegrate into several other objects.
One class – one module.
General rule: one class – one module. In any case, only strongly related objects can be in one module

DB connection

Special objects, the database broker, are responsible for the connection to the database.

Shape-object link

Bummer, or how a favorite IT project failed
A work object is mapped to each form. Only the terminal form creates a work object. The object can be passed to the form from the outside. A form can pass it to another form. Each form has ShowOnObject, GetObjectFromForm methods. These are virtual methods.

Locality

If a form is called from a module, it, as an object, must be local. This will prevent a possible collision of using one form in parallel.

Exceptions

All interruptions are logged and transmitted to the client.

Coding

All codifications that are potentially modifiable are implemented as codifiers of the {code, name} structure. Persistent codifications (days of the week, for example) are implemented as enumerated types.

Implementation

Displaying an object tree in the database

The Tip table contains all objects in the same way regardless of their type. The Movement table shows object movement (state changes and method executions) in the same way for all objects.
The TopB object is matched against the Tip table (Top is usually a reserved word in the DBMS, so the name Top table proved awkward), and each specific business class is matched against a specific table in the DB : the Homo<-gt object; the Homo table, etc.
The relationship between subjects is recorded in the Relation table.
When creating objects that are part of an inheritance hierarchy, the following must be observed: there cannot be a terminal record without a head record. This must be kept in mind when manually creating a database – importing a table of banks, for example. To add a Bank entity you have to have an Enterprise entity. To add an entity Enterprise, you need an entity Entity. To add an Entity, you need to have a TopD entity.
Object explorer, operation, and object advancement are implemented on a common (type-independent) level. The object explorer works the same for all object types. At the concrete level, forms specific to each object work.

Object Explorer

These are forms that allow you to navigate through the object type tree, showing a list of specific objects belonging to the active node of the type tree.
The "State" panel provides additional filtering by object states.

State engines

These are the forms that allow you to advance the state.

Transactions

Operation handling is unified by the fOperation form.
From this form you can exit to the specification of transaction objects and to the payment of the transaction. Example of exit to the payment by the button "Specify transaction payment":

Explorer of operations

Similar to object explorer.
The specific operation can be pulled from the explorer and viewed by the Individualize button.

Access

Management of access to business objects is done at the business level. Access can be by permission or by denial. Access can be :

  • To business object types (Denied or allowed access to all instances of a specified type and all methods of an object of that type)
  • To methods of a type (access to the specified methods of all instances of this type is denied or allowed)
  • To instances (Access is denied or allowed to specified instances of a specified type and all methods of an object of this type)
  • To instance methods (access to specified instances of a specified type and to specified methods of an object of that type is denied or allowed)
  • To forms

Solution structure

The solution is a collection of projects (Visual Studio terminology). There are about 200 projects in the solution. All the projects are bundled like this :

  • Data projects. Each D-type corresponds to a D-module.
  • Business projects. Each B-type corresponds to a B-module.
  • Communication projects. Each C-type corresponds to a C-module.
  • Presentation-projects. Each P-type corresponds to a P-module.
  • Interface projects. Modules contain descriptions of interfaces.
  • Technology-projects. Contains utilities.
  • Infra-projects. Contains infrastructure solutions.
  • Resource-projects. Contains project resources.
  • Service-projects. Service projects are implemented.
  • Gate projects. Act as a gateway between the service and applications. The unicast version does not need a gateway.
  • Application-projects. Contains specific applications.
  • Test projects.
  • Delivery-projects.
  • Site-projects.
  • Addin-projects.

Mapping the solution structure to the Os directory : The OS directory of the solution repeats the structure of the solution.

Application structure

Bummer, or how a favorite IT project failed

Start program

The head program authenticates the user. All interfaces are created in the header module. That is, the headend glues together the implementation and the declaration of the interfaces. The application manager is created. The interfaces are passed to the application manager. The dispatcher creates forms and passes the appropriate interfaces to them.

Implementation of distributed processing

WCF service

Some feature sets can be implemented as services – collaborative applications with a clear API. The result is an application that runs in multiple address spaces. These spaces can be located on any computer on the network. The communication between them is realized by gateway modules. They are located in the client application.
Gateway tasks :

  • Service-to-customer traffic processing :
  • Receive C-object via API with service
  • Convert C-object to D-object
  • Convert D-object to a client
  • Process traffic from client to service :
  • Receive D-object from client
  • Convert D-object to C-object
  • Convert C-object to a service with API

Service Task :

  • Handling traffic from client
  • Accept C-object
  • Convert it to a D-object
  • Pass a D-object to a functional part via local API
  • Traffic processing to the client
  • Receive D-object from functional part by local API
  • Convert it to a C-object
  • Pass a C-object to the client

Web Services

Everything is the same as for WCF with the replacement of the WCF-service by the WEB-service. The latter is a client over the WCF-service, so that the other WCF-clients do not notice the WEB-service.

Specifics

Accounting objects

Accounting accounts are the primary source of analytics. The main object of analytics is an indicator. Its description includes the code of the accounting object to which the indicator corresponds. Therefore, it is extremely important to construct a set of accounting object codes well.
Accounting objects form a tree of accounting objects. Its maintenance is available to the user.

Type tree

All solution types form a tree. This tree is mapped to the Tip table. The Id of the record is the TypeId of the type. All objects are represented in the Tip by records subordinate to an object type record. For example, Homo object records are subordinate to Homo type records – records with Id=Homo.TypeId. This solution allows you to create a universal object explorer that displays the type hierarchy.

Motion

The movement of an object instance is a state change. The movements of all objects are defined in the same way by the Movement object and are stored in the Movement table.

Relationships

Explicit relationships are those that are supported by the information model. For example, the relationship of subordination between subjects.
All implicit object relationships are fixed in the Relation table when a relationship is established. When a loan agreement is entered into, a loan relationship must be recorded. When there are changes to the chart of accounts, a Change relationship is established between the Chart of Accounts document and the changes to it. It is not explicitly present in the model. Making each relationship explicit would overload the model.
You can specify a relation by calling the Relationship form.

Transactions and Integrity

All instances of any objects(including subjects) have a unique identifier. The scope of uniqueness is the database at all times of the project.
All script instances have a unique identifier – the script identifier. A scenario is a sequence of imaginary operations. All instances of any operation have a unique identifier – the transaction identifier. The scope of uniqueness is all tables at all times of the solution. A transaction can spawn other operations that make no sense without the primary, initial operation. If the initial operation is canceled, all generated operations are canceled. Such a chain of operations is treated as one transaction and all operations in the chain get the same transaction identifier. An example of a chain: Opening an account – signing an account opening agreement – paying for the account opening. This chain connects three objects: account, agreement and payment. Cancellation of account opening cancels both the contract and the payment.
Each transaction is documented as part of a transaction. Each transaction begins and ends with a registration in the logbook.
An object can have linked, attached objects in the CoObjects list. This mapping is stored in the database.

Base subject

This is the entity that owns the system being operated.
If in a bilateral relation or function it is not clear who to start with, you should start with the underlying subject. Examples :

  • Buy operation. The underlying entity buys and its counterparty sells.
  • Transaction Sell. The underlying entity sells and its counterparty buys.
  • Relationship Contract. The first party, the subject, is the base entity. The second party to the contract, the counterparty, is the counterpart of the base subject.

Access and registration

Any function starts with a check in the logbook and ends with a check in the logbook.
Access is differentiated as :

  • Access to types
  • Access to methods of type
  • Access to instances of type

An access condition can be interpreted in two ways :

  • On permission (what is not allowed is not allowed). Example : Id=1 – access only to the object with Id=1
  • On denial (what is not forbidden is allowed). Example : Id=1 – access only to object with sysId<> 1

I wish

  • Apply machine learning. For example, determine the creditworthiness of a customer, determine how reliable a business is. But where do you get a large enough array of real-world examples for training?
  • Put some economic theories of microeconomics into the knowledge base, in addition to the rules for calculating indicators. These are, of course, dynamic theories. I.e. it’s something like :

So what’s the reason for the bummer?

I will try to classify the possible causes of failure :

  • Is this an idea-fix? That’s not for me to judge.
  • Unclaimed?
  • I don’t believe in unclaimed. Demand is almost obvious. We are not going to discuss the irrelevance of pressure, pulse, ROE… in medicine. For example, recently we’ve been hearing about the Balanced Scorecard. And this is a subset of analytics.
  • Lagging compared to similar projects?
  • Poor architecture?
  • Toolkit?
  • "Overkill"?
  • Marketing?
  • Didn’t have a large firm with a historic reputation behind it?

I’m leaning toward a combination of the last three reasons.
One last thing, so as not to count the article as an advertisement for the project, I am ready to give the whole project for free in good hands. This is about 20 MB of source code. Of course, you can laugh there too: it was my first C# project. A lot of modern C# features are missing there. First of all, I would change the event-based approach of early C# releases to a modern, higher level event-based approach. Secondly, you could have thought about asynchronous processing capability.
Finishing and thinking whether to press the "Publish" button, I begin to have vague doubts "Who needs it? And what do you want, old man?" And then I decided not to answer that question…
I wish everyone to never have to face the bummer of their favorite projects. Good luck!

You may also like