
Deciding to rewrite getaddrinfo in rust - azth
http://blog.dkhenry.com/2016/02/17/deciding-to-rewrite-getaddrinfo-in-rust/
======
agwa
How security sensitive is getaddrinfo? The recent vulnerability was actually
in glibc's DNS client library, libresolv, not getaddrinfo itself (but it was
triggered when using getaddrinfo). Therefore, I think that rewriting libresolv
in Rust (and possibly the DNS NSS module) would be more fruitful than trying
to rewrite getaddrinfo. It would also be easier - libresolv is in its own .so
so you could swap the entire thing out, rather than having to patch glibc or
use LD_PRELOAD hacks. (I actually started working on this a few days ago :-)

~~~
yyin
Is ldns vulnerable? Did OpenBSD drop BIND and libresolv in favor of ldns and
nsd, unbound, drill? Might be an interim solution.

~~~
Sanddancer
Yes, OpenBSD dropped BIND, as did FreeBSD.

------
tinco
glibc in total is about 460.000 lines of C. If we're planning on
reimplementing it as a whole that's going to be quite an undertaking. Musl is
an alternative libc that's only about 60.000 lines of C. I don't know exactly
why libc is that much larger, as musl also aims to be a complete and POSIX
compatible libc.

Its getaddrinfo is considerably smaller at 90 lines, and there's no goto's, it
looks comfortably readable at a glance. Maybe a nice blog post would be to
compare both implementations to find out if there's a concrete difference in
functionality and code quality.

edit: I should've looked at the 2nd Google result, it's a complete comparison
of a bunch of libc's:

[http://www.etalabs.net/compare_libcs.html](http://www.etalabs.net/compare_libcs.html)

~~~
pm215
Some of that will be glibc's coverage of a much wider range of architectures
and host OSes than musl (a very back-of-the-envelope wc -l suggests that
somewhere between 20 and 25% of glibc code is in sysdeps/ or ports/sysdeps/),
but that's clearly not the only thing contributing to glibc being bigger.

~~~
belorn
There is also GNU specific functions and macros in glibc. Is musl implementing
_GNU_SOURCE? I know that my own project use TEMP_FAILURE_RETRY macro quite
heavily, among with asprintf.

------
mikegerwitz
I understand why people want to rewrite C libraries and such in Rust.

Software like glibc is battle-tested---it is _widely_ used on hundreds of
thousands of systems around the world, and has been used (though not at
today's massive scale) for decades. I understand that glibc is under active
development and there is a lot of new code, but let's keep this in
perspective:

Writing a new implementation of a system is a huge opportunity to introduce
bugs. There is focus on these specific problems, but in the broader scheme of
things, glibc is remarkably stable, performant, and feature-rich. A new
implementation will have bugs, and those bugs might be less likely to be
caught simply because the system will not be as widely used for quite a long
time. Even formally proven systems don't address flaws in the actual program
specification. (See "The Limits of Correctness" by Brian Cantwell Smith for a
good discussion). Also relevant (which I'm reminded of in part because of his
recent death): Peter Naur's Programming as Theory Building.

So even _new_ code to glibc has the benefit of a huge community of both
developers are users to eyeball it and test it out in production on a huge
number of systems.

So rewriting glibc may solve certain problems, but it's bound to create a
whole lot more, considering the narrow range of issues that are being focused
on. New Rust code will have undiscovered issues too, even if they're not
memory or stack related. I feel that this effort might be better spent fixing
and finding problems with glibc---and continuing the development of tools to
find those problems, to benefit _all_ of our old C libraries and programs---
than rewriting for the sake of rewriting.

~~~
wvenable
> Software like glibc is battle-tested

This seems a bit counter-intuitive given that serious problems are still being
found years or even decades later. Perhaps battle-tested is insufficient.

And while new code is likely to have bugs, assuming it's already based on the
original C code it would be less than if it was re-written from scratch based
on a spec. New code can take advantage of the battle-tested nature of the
original code even if it's in a different language.

~~~
mikegerwitz
> This seems a bit counter-intuitive given that serious problems are still
> being found years or even decades later. Perhaps battle-tested is
> insufficient.

Bugs will always exist. But that statement will apply equally to any software,
regardless of language.

> New code can take advantage of the battle-tested nature of the original code
> even if it's in a different language.

That's why I referenced the Peter Naur paper on Programming as Theory Building
---in practice, that may very well not be the case.

~~~
dikaiosune
> Bugs will always exist. But that statement will apply equally to any
> software, regardless of language.

I'd be curious to see some research suggesting that the prevalence and
severity of those bugs is identical regardless of development
tool/language/runtime, but I'd be surprised if that's demonstrable.

The question at hand is not "will safer languages eliminate all bugs?" It's
"will safer languages reduce the prevalence and severity of important classes
of bugs?" I'd wager it's probably yes, but even if you disagree, I don't think
that it's reasonable to suggest that because there will always be bugs we
should never improve.

~~~
mikegerwitz
> I'd be curious to see some research suggesting that the prevalence and
> severity of those bugs is identical regardless of development
> tool/language/runtime, but I'd be surprised if that's demonstrable.

I intended to convey that software written in any language will have bugs, not
that all languages will produce the same types of bugs.

~~~
Manishearth
Yes, but all that gets you is that no language is a panacea; bugs always
exist. It does not address the question of whether or not there will be a
propensity for more bugs (or more severe bugs) when comparing two languages.

~~~
mikegerwitz
> It does not address the question of whether or not there will be a
> propensity for more bugs (or more severe bugs) when comparing two languages.

My argument is based on the act of rewriting it---regardless of language. Many
languages provide excellent guarantees, but that does not protect against bugs
in the implementation itself (logic).

~~~
Manishearth
Yes; and the rewrite can take into account the logic used in the old code
(especially in the security-critical areas) as well as all the vulnerabilities
that have happened before. You're not starting from a complete blank slate;
you can pick up the lessons learned.

Despite being "battle tested", all of these C programs continue to have both
memory and logic errors. I think a rewrite would have the same rate of new
logic issues after the initial code review and testing. "bugs will always
exist" \-- sure, so if we have something that eliminates a class of bugs, why
not use it? The other classes of bugs will be there (and probably in the same
force) whether you rewrite or not.

A lot of these bugs get _introduced_ due to cruft in old code as well. So
there are a bunch of tradeoffs here.

------
lmb
The comment about changing the const pointers is subtly wrong. There are
pointers to const memory and there are const pointers to memory. My favourite
little known C fact. See also
[http://stackoverflow.com/questions/1143262/](http://stackoverflow.com/questions/1143262/)

~~~
delroth
That's not really "little known". Almost every single piece of code that does
string manipulation in C uses that fact:

    
    
      const char* s = ...;
      while (*s++) { ... }

~~~
lmb
I think you have proven my point. I'm talking about this:

    
    
      void something(const char* const s);
    

Maybe I'm extrapolating here, but it surprised me.

~~~
dcohenp
That is indeed what the post you're replying to meant. Note that the pointed-
to value is a constant string, yet the pointer itself is being modified.
That's a fairly common idiom in C string-handling code, only possible because
it's a const char _, not a const char_ const.

------
Someone1234
Is there a good reason why a binary compatible glibc couldn't be re-written in
a better language? People are always going to be using C/C++ and there is
little that can be done about that exposure, but at least the libraries they
rely upon could be better engineered to have certain assurances about safety.

~~~
lmm
You'd lose most of the safety. E.g. a lot of the value of Rust comes from
ownership tracking, but you can't track ownership across a C-ABI boundary
because the information simply isn't there. And I'm not sure assurances in
libraries used by unsafe languages are really that valuable - it'd be like
locking your back door and leaving your front door open. Better to make a
clean break.

------
minitech
Minimalism comes in many forms. Here, we see an example of a content-free
article.

> fun aside, name and service are both const char* so I find it funny that
> they are set in the program, I do understand that from the callers
> perspective they don't change, but still bad form

… are you sure you know enough C to do this?

------
MichaelBurge
I know that it's used for more than just C programs, but it still seems a
little perverse to rewrite the C standard library in something other than C.
Imagine if someone had a preference for writing his Python libraries in Perl
that emitted Python bytecode. Or if C++'s Boost was the output of a Haskell
program.

~~~
Santosh83
I get what you mean, but I guess calling it the "C Standard Library" is really
a historical holdover from times when the entire system was written only in C
(with the exception of a smattering of assembly). In the current context it
should probably be called the "System Standard Library" and when you consider
it like that, a system standard library in another language isn't all that
weird. They all compile down to machine code anyway, so the Python-Perl
analogy doesn't quite match.

~~~
pjmlp
UNIX is the C runtime, kid of.

They just didn't want to force it into other OSes when ANSI C was defined and
eventually POSIX took up that role.

------
saghul
Another potential source of inspiration is Musl: [http://git.musl-
libc.org/cgit/musl/tree/src/network/getaddri...](http://git.musl-
libc.org/cgit/musl/tree/src/network/getaddrinfo.c)

------
chillingeffect
So many tentative posts here :)

I say, "Go for it!" Kozlowski's attitude is exactly what we need. That's not
saying it will ship in Ubuntu 18.04... It will be carefully vetted just like
any piece of software, but it's got to start somewhere and I like that he's
sharing his thought process with the community.

As people have pointed out, it may not actually be the Rust part of it that's
valuable, but the organization and code quality. Rah rah!

~~~
oconnor663
I'm not so sure about the attitude.

> After reading about the newest glibc vulnerability, I have decided to see
> how much effort there is in rewriting parts of glibc in a safe language.

If I were a C programmer, especially if I were a C programmer who worked on
glibc, I might feel a little bit insulted that remark. Safety is a complicated
concept, and calling Rust "safe" and C "unsafe" oversimplifies things,
_especially_ if your Rust code includes unsafe blocks. See also
[http://www.tedunangst.com/flak/post/heartbleed-in-
rust](http://www.tedunangst.com/flak/post/heartbleed-in-rust).

I don't think we have anything to lose by being as diplomatic as possible with
these sorts of things. We can acknowledge that Rust's safety is a work in
progress, and that we're still learning about the drawbacks.

~~~
betenoire
it might oversimplify it, but there is truth in there too

------
agentgt
Has anyone successful linked to a Rust library from another language (ie
Python). I know Rust offers FFI and will apparently make its data
representations C like if annotated and I think has some ABI.

This is the second post I have seen in the last couple of days of "glibc is
bad lets rewrite it in Rust". However the problem is not that glibc is just
bad but rather everything links to it. If other languages can't talk to Rust
easy this would be a nicer problem to focus on first (before rewriting glibc).

I guess this is sort of how its done: [http://siciarz.net/24-days-of-rust-
calling-rust-from-other-l...](http://siciarz.net/24-days-of-rust-calling-rust-
from-other-languages/)

Rust -> C <\- Other language. You have to create the C header files (I'm
surprised Rust compiler or something similar doesn't automate this).

~~~
Manishearth
You don't necessarily need C header files. Rust is able to export symbols
easily with the C ABI; if your language has the ability to take a library and
call such symbols (Python/Ruby do; for example), you're fine. It's only C++
where you need a header file because that's how C++ binds with any library.
It's not a Rust-to-any-language issue, header files are a Rust-to-C++ issue
only; really, an any-language-to-C++-or-C issue (I think such a tool exists,
too).

~~~
dbaupp
A C header is not needed at runtime, it is purely a way to easily describe the
symbols in a library that exist in a (somewhat) type-safe way. It's the lingua
franca for describing libraries, and tooling (such as Rust's bindgen, or
Python's cffi) can consume them to do the setup/create the information
necessary to call functions from other languages. This is much nicer than
manually trying to transcribe the Rust (or whatever) signatures into your
application.

 _> header files are a Rust-to-C++ issue only_

Or... C? (And of course other languages too, as just discussed.)

~~~
Manishearth
> It's the lingua franca for describing libraries, and tooling (such as Rust's
> bindgen, or Python's cffi) can consume them to do the setup/create the
> information necessary to call functions from other languages.

Hence the "not necessarily" in my comment. The parent seemed to be painting
this as extra unnecessary work; I was saying that a header file is _not_
necessary except for binding to C/C++.

If you want to write a C API that will be called from many languages; you
should have a header file so that bindgen tools can consume it. If you want an
API that will only be called by Python .... not so necessary, you may want a
header file anyway but it depends on the situation.

> Or... C?

Sure, goes without saying :)

------
JustSomeNobody
How many new vulnerabilities and bugs will be introduced by doing this ya
think?

Rust in and of itself won't make things all better.

------
Xcelerate
> fun aside, name and service are both const char* so I find it funny that
> they are set in the program, I do understand that from the callers
> perspective they don't change, but still bad form

Correct me if I'm wrong (I haven't programmed in C++ in forever), but aren't
"name" and "service" pointers to constant arrays of characters rather than
constant pointers to (mutable) arrays of characters? In the first case, you're
saying you have a variable that points to a memory location; the variable
itself can be changed, but the data in memory at that location cannot. In the
second case, you're saying that you cannot change the variable that contains
the memory address; however, you _can_ change the data in memory at that
location.

(And if I remember correctly, I believe there are also such things as constant
pointers to constant memory...)

~~~
mbrock
[http://cdecl.org/](http://cdecl.org/)

    
    
        const char *foo
    

declares a pointer to a char that is constant.

    
    
        const char * const foo
    

declares a constant pointer to a char that is constant.

~~~
teddyh
You know, you could just

    
    
        apt-get install cdecl
    

And then do it on the command line. But hey, let's turn everything into a web
service, with NSA et al. monitoring as a bonus feature.

------
armitron
Not only that, but most people in this thread are STILL missing the multiple
elephants in the room.

First, Rust allows unsafe code in the core language [not simply through FFI as
in other safe languages]. So there's a gaping hole that we know _will_ be
abused (performance! performance!) right there. Even if we grant that this is
not the case, here is the deal breaker:

The Rust approach to security depends on an ecosystem that follows the same
approach. To think that this is more than an utopic dream is to enter cuckoo-
land in my opinion. So we digress back to the castles-built-on-top-of-sand.

There's millions of lines of code written in unsafe languages plus all
mainstream operating systems. The Rust approach will never work in this sort
of environment.

An approach that _might_ work however is unikernels and using a language that
promotes design with failures in mind [and makes it extremely easy to rapidly
rearchitect/rebuild/redeploy]. Erlang is the best example in my view, but
there could be more.

Alas, Rust is not really suitable for this either due to its static nature.

~~~
dang
Your comments in this thread have been inflammatory, condescending, and vague.
That amounts to trolling, whether you intended to or not. You've done it quite
a bit in previous threads, too, which is not good. Please don't do this on HN.

Here is how to stop: (1) take out everything inflammatory ("It boggles the
mind how utterly misguided", etc.) and make neutral statements instead; (2)
take out the personal language ("you just don't get it", etc.); (3) replace
vague grand claims with specific factual statements.

If you do this, you'll not only no longer be breaking the HN guidelines,
you'll also be sharing what you know more effectively, which benefits all of
us.

We detached this subthread from
[https://news.ycombinator.com/item?id=11162577](https://news.ycombinator.com/item?id=11162577)
and marked it off-topic.

------
armitron
It boggles the mind how utterly misguided the "let's rewrite everything in
Rust!" people are when it comes to understanding security and its
implications.

Let me begin by introducing my premise: Rust will do NOTHING for security, it
is not an improvement in any way, and at least as far as security is
concerned, it will lead to at best colossal time waste, at worst actively
damage the cause _by misdirection, making ppl feel warm and fuzzy, providing
illusion of security_.

What??? B-b-but memory safety, I hear you say. How can this even be possible I
hear you cry.

Memory safety (assuming that Rust completely solves it, which it absolutely
DOES NOT) is but one (small) piece of the entire puzzle. In the same way that
the skilled craftsman knows that he can't build a castle on top of sand, the
skilled (and security conscious) software engineer _should_ realize that
memory corruption (and race conditions in the same unit/process) are not the
foundation of the security chain. I dare say that a lot of the bugs exploited
out there today (but not necessarily made public), are pure LOGIC bugs (that
include race conditions) that stem from the interactions between extremely
complex autonomous subsystems. Rust is simply operating at a too high-level
abstraction layer to deal with any of these issues. This is the logical
progression of Sergey Bratus's weird machines taken to the next level.

Some examples:

The Linux kernel. Any UNIX operating system taken as a whole. Any Windows
taken as a whole. Any OSX system.

Anything built on top any of the above. Anything built on _combinations_ of
any of the above. Weird machines are not limited to isolated units, you know,
they very well exhibit EMERGENT behavior.

Are you starting to get the picture now?

And then of course we _still_ have the bugs that stem from unsafe languages,
as long as there's a path that cuts through all the intermediate layers of
"safe" code. Graphics drivers and webgl anyone? Truetype in the kernel?

Rust has plenty of things in its favor but to position it as a cure for issues
that it doesn't even know how to address is a colossal mistake. We have given
rise to a beast that threatens to consume us all. It's one thing to despair
due to the immense complexity of the domain and quite another to step into an
imaginary realm and solve illusionary problems that we create just to feel
"safe".

~~~
topspin
There were about 36 000 traffic fatalities in the US in 2015. This is inherent
to the trillions of miles traveled in affordable machines operated by drivers
of variable competence, and nobody has a solution to fix the problem today.
Maybe one will emerge, but between now and that day we're going to keep
driving and keep killing each other.

Should engineers, admittedly lacking an absolute solution to traffic
fatalities, forego seeking designs and materials that reduce the frequency and
consequences of failure?

Rust delivers a higher degree of memory safety. It is a better tool in the
tool box and arguing that engineers should ignore it because their efforts are
inherently flawed won't work, any more than arguing that a new headlamp or
traffic light design is futile.

Also, you should understand now what you're up against. The legacy stack is
riddled with flaws; every day we face a deluge of security notices, some large
fraction of which are caused by memory safety problems. In other words, your
opponents in this debate have an endless supply of ammunition.

Good luck with that.

~~~
Alupis
> here were about 36 000 traffic fatalities in the US in 2015

> nobody has a solution to fix the problem today.

You're entire premise revolves around 0.01% of the population. That's not a
good basis.

There's about 34,000 automobile related deaths annually in the US[1]. There's
about 323,000,000 people in the US.[2]

The point I'm making, is let's not get carried away with exaggerating how
"bad" a "problem" is. According to the CDC, more people die annually from
accidental poisoning (about 39,000) than automobile related accidents - but I
don't hear all the calls to "solve" that problem.

Will things be "safer" written in Rust? Maybe. Does Rust solve everything?
Absolutely not. Yes, a lot of bugs are "unsafe" memory access issues, but a
lot of bugs are not. Re-writing everything in a pet language that happens to
be popular today in 2016 is not a good call. Rust is fairly new, and nobody
has any idea if it will stand the test of time. C has, clearly.

Instead of getting out our pitchforks, we should rally to guarantee these
critical projects are fully funded and staffed with full time developers. That
alone, will yield far greater results than some lofty goal of completely re-
writing 100's of thousands of lines of code which have been working, tested,
and matured for almost 3 decades.

[1] [http://www.cdc.gov/nchs/fastats/accidental-
injury.htm](http://www.cdc.gov/nchs/fastats/accidental-injury.htm)

[2] [http://www.census.gov/popclock/](http://www.census.gov/popclock/)

~~~
pcwalton
> Yes, a lot of bugs are "unsafe" memory access issues, but a lot of bugs are
> not.

A huge number are. For example: [https://www.cvedetails.com/vulnerability-
list/vendor_id-72/p...](https://www.cvedetails.com/vulnerability-
list/vendor_id-72/product_id-767/GNU-Glibc.html)

> That alone, will yield far greater results than some lofty goal of
> completely re-writing 100's of thousands of lines of code which have been
> working, tested, and matured for almost 3 decades.

Windows, the popular Web browser engines, and antivirus software are all
examples of systems software that are very well funded. But they still
regularly fall to memory safety problems. The converse is also interesting to
look at: djbdns and qmail are examples of poorly funded software, but they
have had very few memory safety problems.

I agree with the general principle that funding helps security overall,
though.

------
akerro
Please call your implementation getaddrrinfo

------
w8rbt
___Some people, when confronted with a problem, think “I know, I 'll rewrite
this in another programming language.” Now they have two problems._ __

~~~
maxerickson
This is a great story to make glib comments on.

~~~
w8rbt
This glib comment has a long history in technology. So it seems appropriate.

    
    
        http://regex.info/blog/2006-09-15/247

~~~
steveklabnik
I thought your parent was making a pun about 'glibc' and 'glib'.

