Home .NET Asynchrony in .NET, popularity on Stack Overflow, “church” software: an interview with Stephen Cleary

Asynchrony in .NET, popularity on Stack Overflow, “church” software: an interview with Stephen Cleary

by admin

Asynchrony in .NET, popularity on Stack Overflow, "church" software: an interview with Stephen Cleary
Stephen Cleary enters the top 100 Stack Overflowusers. Mainly because of his answers related to asynchrony in .NET. His life is not limited to programming: he tweets himself "Christian" first and then "developer".
Now, in connection with the appearance of async streams, his knowledge is especially relevant: as a speaker on such a topic, Stephen’s figure just begs to be heard. And very soon on DotNext he will talk about it and will tell In the meantime, we asked him about it all : religion, Stack Overflow, and asynchrony.

Biography

– The first question is simple : tell us about your professional background.
– I got interested in programming quite early, I was about 7 or 8 years old. Then at school I was able to write a little program in Logo, and that was my first programming experience. Later, when I was about 12 or 13 years old I bought my first computer. And when I chose Computer Science for my college application, it was a pretty obvious decision: I had already been interested in computers for some considerable time.
In 1998, when I graduated from college, I started working for a local company that did conveyor belts, automated industrial vehicles, and other factory automation. Seven years later, the company moved to Detroit, but I did not want to move there, and since then I have been able to work in different companies. I’ve been working remotely since 2013, and I’ve been working at Faithlife for the last year and a half.
– It’s funny that you started with Logo, and now you’re at a company whose main product is called Logos. What exactly do you do there?
– Backend : web services that are accessed by Logos and other company products as well. Faithlife has a number of church-related products. Logos is for Bible studies. And then there’s Proclaim, which is designed for those who give sermons (helps with slides and the like). Historically Faithlife has made products for pastors, but now they’re adding things to help regular people.
I do backend, I often have to deal with ASP.NET Web APIs there, and right now I’m working with Docker. Right now we have some in the cloud and some on-premise, and some things shouldn’t be moved to the cloud at all, but some things we want to move there – and I’m doing that.

Religion

– You don’t often see a company that makes "church" software, so I would like to clarify: does the work on the backend there look like everywhere else, or is there a specificity? And the developers are usually believers and use the company’s products themselves, or not?
– There are no fundamental differences in the way we work: you have to think about the right API design here as much as anywhere else. Some of our products (especially the desktop ones) have been around for a very long time and you have to deal with old APIs. And when starting to use Docker, it’s important to think about backward compatibility. All in all, we have about the same concerns as others.
Although Faithlife makes products for the church, it is not a Christian company. You don’t have to be a Christian to get a job here: first, such a selection would violate American law (it would be religious discrimination in hiring), and second, the company wouldn’t want it anyway.
But as for the developers’ use of the company’s products, it’s encouraged in every way possible. For example, we recently held a training day on how to use Logos. We also have a lot of internal tools – in which case it’s encouraged to go work a day in the department that uses your tool to see firsthand how it goes.
– Your Twitter feed has the words "Christian" and "developer" in the "bio" box. Do these two parts of your identity intersect somehow?
– I would say separate. I’ve always tried to separate my work life from my home life by not doing work in my spare time (although it hasn’t always worked out). Church is a big part of my personality, most of my friends are from there. I haven’t made many friends at work.
– Well, your bio still says "addicted to writing OSS" and giving people free software sounds altruistic and Christian. Is there no correlation in this case?
– Hmm, maybe. I think from a philosophical point of view one can really see a correlation between open source and Christianity: both emphasize the "giving to the people" thing you mentioned.
There is also a "humanitarian" kind of opsource. I’m not involved in this kind of stuff myself, I have general purpose libraries. But I know Christian developers who have worked for free on something that directly affects other lives, especially in places with poor populations. For example, software that makes sure fire alarms work properly. There’s a correlation here, of course.
– You’re also active on Stack Overflow – is that a way for you to unselfishly give something to the community, too?
– I wouldn’t say that, it feels more like "teaching" to me. Although "teaching" can also be related to Christianity… There have been many teachers in my family, and I continue it in my own way-not as a profession, but through Stack Overflow and conferences. It’s how I meet my need to do it.

Stack Overflow

– We have a number of more questions about Stack Overflow, the first stupid one. You recently got a badge there for your answers on Windows Phone 8:

So this just happened… #soproud pic.twitter.com/fj50IjEkoe

— Steve Cleary (@aSteveCleary) August 23, 2019


How is it that you got a badge in 2019 for answering about a virtually dead technology? Is anyone still asking questions about it?

– To get a badge, you usually need at least a certain number of answers with a certain number of votes for those answers. I assume I got this badge because now someone has voted for some answer I posted years ago, and it finally got the right number of votes with it. It’s funny because I haven’t seen a Windows Phone 8 question in a very long time! (laughs)
– At SO your reputation is 320, 000 – more than a quarter of that of the legendary John Skeet, even though you’ve given an order of magnitude less answers than he has. How did you manage to accumulate that?
– I’m not entirely sure myself. Reputation is easier for those who came to the site before everyone else, and even though I was among the early adopters, I still came later than most. I started answering questions – 1-2 a day at first, which is crazy far from John Skeet. Focused on the topic of async/await and concurrency and multithreading – simply because that essentially became my specialty then. I was already doing asynchrony many years ago, and it was a pain back then. So when async/await came along, I saw right away what their point was. So I was in the right place at the right time, and my "teacher’s blood" helped me explain everything in a way people could understand. And in the case of Stack Overflow, I turned out to be a local async/await expert. And John Skeet – well, he didn’t say it directly, but I assume so – leaves most of the async/await questions to me. But he certainly answers a lot more than I do, you can’t catch up with him!
– Looking at the reputation numbers, it’s easy to think "it takes a person half their life to answer" – but how much time do you actually spend on SO?
– Not so much. I check Stack Overflow a couple of times a day, and usually answer 1-3 questions. And most of the positive votes I get for answers left years ago. That’s how SO helps those who came in early : they answered the questions first years ago, got votes, and the votes make those answers more visible, and they end up getting new votes as well. It’s an avalanche effect, encouraging old answers.
I think this causes some problems, because sometimes there are new answers that refer back to more modern technology and therefore better, and the old answers are not updated. But, in general, I get most of the votes for the old answers, but I still try to answer the new ones every day. I don’t spend a lot of time on it, I just do it all the time over the years.
– When John Skeet came to visit us at DotNext, we asked him about the current state of Stack Overflow and what he thinks are the problems with the site. Interesting to compare : what do you think about this?
– I think SO is making improvements, especially in the last year. They are now trying very hard to improve the quality of the questions. Often people asking something on this site for the first time are unfamiliar with how to properly ask technical questions.
And it’s a problem that’s always been present on the Internet. In the ’90s, when everyone sat in newgroups, it was there, too. Ten years later, everything was happening in mailing lists and Google Groups. Now the new resource for questions about programming is Stack Overflow. But there have always been questions about programming, and how to ask a good question has always been a problem, and how to maintain a friendly community has always been a problem, too. Perhaps these problems will never be completely solved. I’m not saying we shouldn’t work on them-we certainly should. But if you look back in time, there were tutorials on how to ask technical questions, even in the 90s.
There are a couple of problems associated with newcomers to Stack Overflow. To begin with, many of them have never answered someone else’s question before asking their first one. So in many cases, they just don’t realize all the things that need to be presented in a question to be answered.
And then, imagine you’re working on a problem. You’re up to your ears in code, your head full of it all. And you’re looking at some particular thing that doesn’t lend itself in any way. It’s a small specific detail of a big system, and usually you’re asking, "How do I do this small thing?" without realizing that you have all this context you know, and you’re not putting it into the question. Often the correct answer to the question, "How do I do X?" is "Don’t do X, do Y." This is a trap that people often fall into when composing their first questions. They don’t realize that their answer in its current form is "unanswerable."
And aside from the question quality issue, there’s also a tendency – especially on Stack Overflow, where everyone is fighting for points, trying to answer as quickly as possible – to promptly close the question or quickly scribble not the nicest answers. I don’t mean "spiteful"; I’ve seen very few outright spiteful comments – only a few over the years. Rather, they are harsh, and to the author of the question, it reads as unfriendly.
Stack Overflow has taken some simple steps to fix this. Now when people first ask a question, the site tells them what to include. They previously added "look at these similar questions, " which was a good first step. And now there’s a whole system to go through for the first question that helps put it together well.
They also add reminders for people who write responses. Like "This is a newbie, be friendly, " and it’s a good way to remind them that most people don’t write questions well the first time, and that’s totally fine. In general, it’s being worked on. Will this problem ever be completely solved? I doubt it. But progress is possible.

Asynchrony

– In post on the 10th anniversary of the iPhone, you write that its advent affected asynchronous programming because mobile apps are required to be responsive. And for those who are new to development and haven’t seen a world without async/await, can you give a general historical background on the development of async?
– Well, asynchronous programming has always been possible. I think it was already being done in the 60s. And it had the same advantages for a long time: a more responsive UI and a more scalable server part. The problem was always maintainability: it was very difficult to maintain asynchronous code. It was originally built on colbs.
I think the big step was the emergence of Promise.In JavaScript they called it that, and in C# it’s Task or ValueTask. And that was a big step forward, because now we have an object that represents an operation. You can keep track of its progress and so on. And async/await in any language is basically just syntactic sugar around Promise.
I would say that the emergence of Promise is the event that affected asynchronous code the most. And also in the case of async/await, it’s important that the state machine is created for you. In the old days, when we were working with colbecs, we had to make our own state machines. And that was always difficult, because inevitably you would forget some state or transition, and nothing would work. And it was very difficult to debug. Async/await didn’t bring the ability to write asynchronous code itself, but the ability to write code that could be maintained.
– Do you now see the situation as having settled down, or can we expect more changes soon?
– I think it’s pretty stable now. Async/await looked revolutionary-but only to those who hadn’t done asynchronous programming before. And for those who did, it felt very natural. But, all in all, it was a big deal.
Now another development comes with .NET Core 3. They’re doing what everyone calls async streams, even though they’re actually asynchronous enumerables. I think it’s going to confuse people because there’s already a Stream type that has nothing to do with async streams. In general, there will be more incremental improvements. Will we see something as massive as when async/await first announced itself? I don’t think so.
If you want a new paradigm shift, there’s always the possibility of an actor-based system. Or something like gorutin, where all asynchrony is implicit. I think these things are similar in some ways. The problem is that you can’t add something like that easily in .NET. I think there are too many limitations for .NET to go for that, so it’s unlikely to happen. If we see a massive move to an Actor world or a goroutine world, where the approach to concurrency is not the same as with today’s multithreading, it will require entirely new languages and ranting. And .NET is not worth going for that. And I don’t think programming as a whole will make that leap. I may be wrong, but my current position is this.
– More on the question of how much things change over time. Many books about programming are rapidly becoming obsolete, but where there is less change, they stick around longer. What about your "Concurrency in C# Cookbook." , how much does it require updates?
– Good question. I just recently published the second edition, and the first one came out in 2014. So it’s been five years and we’re already on the second edition – that seems like a quick development to me.
I think it also depends on how the book is written. I’ve tried to write it so that it stays up to date as long as possible. You just have to try not to refer to things like Windows Phone 8. But it will inevitably become obsolete anyway. Async streams and stuff like that come up, and I wanted to include stuff like that in the book. Something ended up being outdated, but most of the material moved into the new edition unchanged.
And, of course, it all depends on who the book is aimed at. Of course, a book on using Visual Studio 2008 will become obsolete very quickly. But I think there is a place for true classics. I consider "Code Complete" one of the best books on programming in the world. And how long ago was it written? I don’t know. Decades ago. And it’s still a fantastic book! Something in it is outdated, but overall it’s still great.
– Recently on Twitter were burbling about async/await, starting with a tweet by Vaughn Vernon, author of several books on DTD and the actor model :

Latest update on @dotnet and blahAsynch(…) and await.
It leaks EVEYWHERE.
It is EFFECTIVELY BLOCKING.
It’s nearly impossible for a toolkit to NOT LEAK it EVERYWHERE once it is used anywhere.
I hate this idiom/API even more than ever.
We are going to find a way around this.

— Vaughn Vernon (@VaughnVernon) May 19, 2019


He doesn’t like the intrusiveness of async/await – in his opinion, it spreads all over the code and can cause traces to be blocked. What do you think about this? Should something have been designed differently?

– Yes, async/await is probably the most common complaint. There are a couple of things I’d like to point out here.
First, asynchronous code, to be truly asynchronous, requires asynchrony from everyone who calls it. And there’s no getting around that. Whatever paradigm you use (even one of the old ones), you’ll end up running into that. I had a paper where I chronologically go through all the paradigms and show how the code got cleaner and better.
There are a couple of different approaches to avoid the ubiquitous async/await. The first is to isolate all I/O into separate objects. You can use design patterns like Ports Adapters: this allows you to keep all the I/O outside of your business logic, and then it won’t need any async/await at all. I recently saw a whole paper on how to prevent "async everywhere" by refactoring the design so that the business logic never deals with I/O. I would recommend this approach.
There is another approach to dealing with async/await, but it’s not feasible in .NET. I think Go tried to do it with goroutines. In fact, everything there is all async/await. You can remove async/await from the language by adding it to everything there and making it implicit.
There is no other way to do it. When async/await appeared, someone (I don’t remember who exactly) said that it’s like a zombie virus : as soon as part of your code is infected, it starts spreading.
– Some developers use the actor model, explaining that it’s easy to manage threads (actors are essentially single-threaded). Do you think that with the right choice of libraries, developers can get rid of the complexity of programming concurrency?
– Well, not quite. Because even with the actor model, there are still problems. You won’t run into race conditions like you would in a multithreaded program, but you will have some friendly difficulties.
For example, problems with message delays or asynchronous messages. These are usually needed to prevent deadlocks in the actor model. But any actor model that uses asynchronous messages can also cause deadlocks. Asynchronous messages can also create coordination problems of their own. Usually you also have to deal with idempotency at this level.
Also, using actors with asynchronous messages can make state management pretty messy, unless it happens entirely in messages. And even then, you’ll have trouble with the state. In general, from my point of view, the actor model cannot solve all problems. I would describe it this way : it can only change where exactly the problems will occur.
– You’re known as a dotnetter, but you use other languages like TypeScript. When you try them, how does that make you look at C# and .NET? Do you encounter anything there that you’d like to see in C#?
– Great question. C# took a lot from other languages. One of them that it took a lot from was Python. And I like that. I haven’t written in Python in years, but I think it’s a very well-designed language. I really appreciate the enumerator blocks that C# borrowed from Python. And Python was also one of the first languages to have Async streams, so you could say C# took that from there too.
I like different languages differently. I generally prefer static typing, so I haven’t been using Python much lately, and I use TypeScript instead of JavaScript for the same reason. JavaScript has its advantages simply because of its single-threadedness. For example, if you look at the relevance of Reactive Extensions, you’ll see that in .NET it didn’t take much. And in JavaScript, you can see Rx everywhere.
– The last couple of questions are about your talk at DotNext. You’re going to talk about Asynchronous streams there – can you briefly tell .NET developers what to expect and why they would benefit from this talk?
– So, my report about asyncronous streams: why they were added to the language, and what is the main scenario for using them. I’m approaching this very pragmatically, and I don’t go into all the details of what’s going on under the hood. There are things you need to know to use async streams properly yourself, and there are things you need to know to consume async streams in your code, and you’ll have to do that as they come on the scene.
– One last question. You are very difficult to get to the conference – why did you decide to come to DotNext?
– Yes, I’m not the type of person who goes to conferences all the time, I’m a developer first and foremost. I think this helps me as a speaker: I’m very pragmatic and in my reports I give things which I could use myself.
Your program committee contacted me about a year and a half ago, and I initially didn’t mind coming, but needed it to fit my schedule. And this year, because of the async streams coming out, all the conferences want a report on them, so I decided to speak at a little more than two conferences a year as usual. Glad to be able to be at DotNext because of this.

DotNext 2019 Moscow will be held on November 6-7. In addition to Stephen Cleary’s talk, there will be dozens of other talks for dotneters – you can see full program

You may also like