From the dynamic languages the SBCL compiler is something you might want to try.
The compiler tells you the usual (?) stuff like missing args, wrong named arguments, missing functions, undefined variables, syntax errors, etc.
But Common Lisp has also a (relatively primitive, compared to something like Haskell) type system and the SBCL compiler can make use of type declarations (for compile time type checking and for runtime type errors) and does also do some form of type inference.
With SBCL you get all the interactive dynamic features and a compiler which you can use to check your code and make it faster. The compiler can for example tell you which operations it could not optimize and why. Then you can decide if it is worth to put some effort into improving the code - for example by adding better type declarations. Common Lisp also lets you inline code or have some data be stack allocated...
Common Lisp also has a sophisticated error system, which allows you to get lots of excellent error reporting and handling.
Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB.
What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks.
Common Lisp has very little UB, so the C strategy is harder to execute.
SBCL basically first needs to sucker their users into overusing one corner of the language that does have UB, type declarations.
They get their benchmarks, at the cost of saturating the ecosystem with code that overuses UB, basically poisoning the ecosystem.
Please don't take HN threads into programming language flamewar; we've all seen what this leads to elsewhere. It's both possible and much preferable to simply make your points flame-free.
That's not at all what SBCL is doing. Each type declaration is treated as an assertion, and only then it performs optimization. And there is no undefined behavior at all.
And even without type declarations SBCL is actively deriving types and optimizing things that are proved to have a restricted type.
Spreading misinformation like this is what's poisoning the ecosystem.
> Spreading misinformation like this is what's poisoning the ecosystem.
Please don't respond to flamebait by upping the ante. We've all seen what programming language flamewars lead to. Your comment would be much better without the last sentence.
Well, there is. Violating type declarations is UB in Common Lisp, the only thing the standard says about it is "the consequences are undefined". In SBCL, an error will be signalled, but there's no guarantees about what other compilers might do. This could show up if a Lisp programmer who primarily used SBCL was relying on the type system to do some of their input validation for them when portable code would have to actually check it (ie, in their head they might think their program will raise an error if you pass it garbage, when really it just invokes UB if you pass it garbage). I still don't think it's that big of an issue, and mentioning it alongside benchmarks certainly indicates some misunderstanding, but it's not completely baseless.
That is theoretically true, but what implementation really does that with SAFETY being 3?
Claiming that SBCL following the standard is poisoning anything is completely baseless, though.
Sure, but the point is that that exact same code will throw a type error in SBCL. It at least complicates what you need to do before you can say "what implementation doesn't behave like that?".
But what is the point, really? SBCL will disable type checks with safety=0, LW will enable with safety=3, debug=3. Is the point "read your documentation before relying on things"?
Or is the OP's point that SBCL shouldn't exist because it's "poisoning" and "a bad thing"?
That's the problem with such FUD, can't ignore it or somebody might actually believe it, can't just say "nonsense" since that's not convincing.
> What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks.
I sort of see what you're getting at, but I think what SBCL does isn't really comparable to the C situation. The complaint people have with UB in C is compiler-writers treating it as a licence to do very unintuitive optimisations; it uses it as a justification for being extremely hostile to the programmer and then the defence is "well the standard said the behaviour was undefined, why would you think there's "obvious" code for the compiler to emit here?". In the SBCL situation, the standard doesn't guarantee any behaviour around that feature, and SBCL is using it as a licence to be more helpful to the programmer, so I think most of the issues people might have with the C situation don't really transfer.
For the great majority of programs, the difference doesn't matter either: breaking type declarations being an error (as in SBCL) vs being undefined (as in portable CL) aren't incompatible, as long as your program doesn't rely on a condition being signalled when you violate a type declaration.
I can see how relying on SBCL's interpretation of the standard would cause problems, but when the difference is that SBCL just changed "the behaviour is undefined" to "it is an error", I don't see that introducing a wave of unportable code.
> They get their benchmarks
Type declarations that can't be statically verified transparently degrade to runtime assertions; I don't think that choice was made to do better on benchmarks.
> at the cost of saturating the ecosystem with code that overuses UB, basically poisoning the ecosystem.
I don't think there's that much code out there that relies on errors being signalled when it violates type declarations as a part of normal execution (quick, link a few projects that make use of this), certainly not enough to say that trend poisoned the ecosystem.
Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax for configuring, you guessed it, safety and speed.
The undefined behavior kicks in when you explicitly defeat safety. (safety 0) (speed 3) means "throw out the life jackets and assume that these declarations are true, making the code as fast as possible".
If you don't have the confidence for this over some piece of code, then just ... don't do that.
Safety is on by default. If some implementation changes that, that is bad.
The C language lacks any means to specify these preferences over a specific scope of code. Some compilers have certain #pragma directives, and beyond that, there are only module-level compiler flags. Those don't specifically give you safety. Usually there is just an optimization level. If optimization is omitted or perhaps specified as -O0 or whatever, that makes code sort of behave more safely in some handwaving manner in that certain "confusing, optimizy things" don't happen.
Also note that Lisp has a defined-order of evaluation. A function call like (f (inc i) (inc i)) is not comparable to the C f(++i, ++i) because it performs the left increment, obtains its value, then the right increment, and obtains its value.
Also note that C requires declarations everywhere. Lisp declarations are optional. Before Lisp declarations can be used in a wrong way that causes a malfunction, they have to be used, period.
We can use Lisp declarations to cause an integer overflow, like in C. If we don't use them, the integer type automatically prevents them by switching to bignums.
In C, even the function which is called once, main, is stuffed with declarations and compiled for speed, often with optimization on in actual real-world projects.
The SBCL people would have a long way to go before they screw up anything to the level of C.
> Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax
To be pedantic, those aren't type declarations. More importantly, the (un)safety of a type declaration isn't predicated on any specific safety/speed settings. The mere use of a type declaration implies that the programmer certifies their accuracy inside its scope; an additional optimize declaration is not required.
It bears repeating: While Common Lisp has "type declarations", they don't behave as they do in statically typed languages. The are certificates, assurances by the programmer to be exploited at will by the implementation.
They are values of optimization qualities and guide the compiler strategy.
> More importantly, the (un)safety of type declaration isn't predicated on any specific safety/speed settings. The mere use of a type declaration implies that the programmer certifies their accuracy inside its scope;
With an optimize quality 3 for SAFETY a compiler will not reduce safety, even though there are type declarations.
The Common Lisp standard REQUIRES that safety at 3 causes 'safe code':
Which usually means full runtime checks in interpreted and compiled code, calls, system calls, ...
> an additional optimize declaration is not required.
Sure you need to have the right compiler qualities set to see those effects of your type declarations.
LispWorks:
CL-USER 4 > (proclaim '(optimize (safety 3)))
NIL
CL-USER 5 > (defun foo (x)
(declare (string x))
(+ x 2))
FOO
CL-USER 6 > (compile 'foo)
FOO
NIL
NIL
CL-USER 7 > (foo 'bar)
Error: In + of (BAR 2) arguments should be of type NUMBER.
1 (continue) Return a value to use.
2 Supply a new first argument.
3 (abort) Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
CL-USER 8 : 1 >
Oh, the LispWorks compiler has ignored the type declaration at safety 3. The runtime checks are still there. The code has the same safety as without a type declaration.
Undefined behavior gives the implementation the freedom to insert any behavior whatsoever (like the proverbial demons flying out of the nose). An assertion isn't any behavior, it is a form of run-time error checking.
Run-time error checking is what the "safety" optimization option means!
From CLHS:
Name Meaning
compilation-speed speed of the compilation process
debug ease of debugging
safety run-time error checking
space both code size and run-time space
speed speed of the object code
(optimize (safety 3)) means "greatest possible run-time error checking". Why wouldn't the implementation insert error checking based on the user's type declaration, when asked to provide the greatest possible run-time error checking?
It is. For example, a failed assertion inhibits effects that should have occurred before the "actual" type error happens.
The only reason SBCL is able to insert assertions into safe code without giving up conforming implementation status is that, as you say, UB allows anything, and violating a declaration is UB even in safe code.
There isn't necessarily an actual type error! Remember, the declaration provided by the user can be an inaccurate estimate of what the real type constraint is at that program node. The program might in fact require a string there, but the programmer's declaration says integer. So the assertion goes off when a string value occurs, when that would in fact correct in the absence of the declaration.
This alteration of behavior by the assertion is not an arbitrary choice of behavior; it is a diagnostic behavior, in line with the diagnostic meaning of safety.
ANSI CL does literally say under "Declarations" that "The consequences are undefined if a program violates a declaration or a proclamation". Yet, it's not reasonable to allow any behavior whatsoever if a type declaration is violated by a run-time situation, in safe code. It just makes no sense at all to allow a crash, or random bits being silently flipped, etc. Given that safe code, in the absence of declarations, catches type errors; why would declarations regress from that, in safe code.
If you re-read everything in this subthread, you'll find a lot of argument from seasoned lispers that I can and should ignore the fact that type declarations are unsafe even at the highest safety settings because of "what implementations do."
If this discussion had taken place before SBCL, the logical conclusion from what these seasoned lispers say would have been that I can and should use type declarations in safe code for documentation purposes (say), because implementations ignore them at high safety, e.g. from lispm: "Oh, the LispWorks compiler has ignored the type declaration at safety 3. ... The code has the same safety as without a type declaration."
If I had followed such advice from seasoned lispers, liberally putting type declarations into safe code for documentation (say), SBCL's new behavior would have burned me, causing aborts in perfectly working code just because I got my "documentation" wrong. Some observations:
(1) SBCL would be allowed to burn me because I invoked UB. Nothing else gives them license to break working code in the presence of a wrong declaration, but they have that license, and I can't sue.
(2) I shouldn't have listened to lispm. Risking UB in safe code because most existing implementations wouldn't have burned me (before SBCL) would have been bad advice. Peppering safe code with declarations does nothing to improve safety, risking UB can only ever deduct from safety.
(3) As a consequence of (2), encouraging the use of declarations everywhere is wrong. To do so, in the words of PuercoPop, is "missing the point of the Lisp's type system." SBCL is wrong to do it.
> If you re-read everything in this subthread, you'll find a lot of argument from seasoned lispers that I can and should ignore the fact that type declarations are unsafe even at the highest safety settings because of "what implementations do."
The word "safe" in Lisp is tied to run-time checks.
"Less safe" means fewer run-time checks.
If a declaration causes a check to be inserted which signals a condition in code that otherwise would not, that isn't "unsafe"; it is something else. "Unsafe" in Common Lisp specifically means that a check didn't take place which normally would take place.
A monotonic increase in the variety of checks performed at run-time cannot be called "unsafe"; that amounts to an incorrect use of the ANSI document's defined term.
You can say that adding declarations to a program is "risky": it creates the risk that conditions will be signaled.
Those extra behaviors are in fact allowed because the program has violated a declaration, which is undefined behavior. That is a very general statement in the standard which is not expected to have malicious interpretations. Declarations are part of the program and can change its behavior; it is not spelled out in detail how, but the optimization parameters like safety and speed have obvious interpretations.
If a declaration is processed for safe code, and a behavior is added other than a diagnostic behavior, that could be characterized as malicious. For instance, supposed the program abruptly stops, without any diagnostic, so that it is not known why. That would not be acceptable. It would not be acceptable for a variable to be mysteriously altered, and for computation to continue, so that a problem occurs later which is nigh impossible to trace back to the violated declaration.
If you think the implementors are malicious, use something else. Malice could be perpetrated in almost any area of an implementation.
Think about it. Suppose you're hoping to use declarations to speed up code. You add the declarations and keep the code safe at first. If declarations are completely ignored, you're not getting any help! The code is working exactly as before and then when you drop safety and add speed, it fails. It fails not in nice ways, but catastrophically, due to reasons that could have been caught had the declarations been processed in safe mode.
> type declarations are unsafe even at the highest safety settings because of "what implementations do."
Type declarations are not unsafe at highest safety settings in most Common Lisp implementations.
Personally I write code for real actual implementations and their documented behavior - and not just for a spec. They implement the pragmatic part of a programming language, extend it with various features. The CL spec for example says nothing about GC - which would make memory allocation 'unsafe' when we follow your arguments - fortunately implementations provide GCs. Similar, implementations provide a mode when safety = 3 where full runtime type checks are enabled.
I for example sometimes exploit that parts of the implementation is using CLOS for built-in functionality where the spec does not require the use of CLOS (streams is an example) - knowing that this code will not run in some - often also uninteresting for me - implementations.
> SBCL's new behavior would have burned me, causing aborts in perfectly working code just because I got my "documentation" wrong.
That's true, SBCL tells you that your documentation is wrong. Which is a good thing. Somebody would read your code and would be confused by your wrong type declarations.
Actually: wrong type declarations for optimization purposes is the real danger. SBCL helps to find the problems.
> encouraging the use of declarations everywhere is wrong. To do so, in the words of PuercoPop, is "missing the point of the Lisp's type system." SBCL is wrong to do it.
That's not what SBCL does. SBCL allows you to add declarations. But also SBCL does type inference, so types get propagated without the need to type everything. Most better Lisp compilers do forms of type inference. I also gave you an example where SBCL takes advantage of method argument lists - thus you don't need to add type declarations, since the method already tells which class the argument is of.
Advice: follow the spec, but understand the broader Common Lisp tradition codified in its implementation.
You are missing the point of the Lisp's type system.
> There has been some confusion about the difference between type checking for the purposes of compiling traditional languages, and type checking for the purposes of ensuring a program's correctness. [0]
Lisp type system is mainly designed for the former. Abstract data types are useful for the later use.
Thanks. Seems my assessment and SBCL's self-description match perfectly, as they explicitly aim to "reward the use of type declarations throughout development" and exhort users to "always declare the types of function arguments and structure slots as precisely as possible".
They do want their users to use one of Common Lisp's least safe features pervasively.
If the point was to mine assertions for type information, they could have encouraged the use of assertions. But they don't. They encourage the use of declarations, which is entirely not the same thing, and it's a bad thing for the Common Lisp source code ecosystem, even if not for the SBCL ecosystem. Hurray for SBCL.
> They do want their users to use one of Common Lisp's least safe features pervasively
It's not an unsafe Common Lisp feature. The standard does say nothing about its unsafeness. Only implementations may be unsafe. The Common Lisp standard says nothing how the implementation deal with type declarations.
There is a wide range of behavior in Common Lisp implementations dealing with type declarations. Some will do nothing with type declarations - thus its not more or less safe if one define types.
Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime.
Some compilers may add less runtime checks and will create specialized code -> less safe.
SBCL provides several ways to deal with that depending on compiler settings.
That SBCL advanced features are a disadvantage is nonsense.
> If the point was to mine assertions for type information, they could have encouraged the use of assertions.
With that in mind:
> Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime.
Users can portably and safely use assertions if they want that behavior. Encouraging the use of declarations instead is neither portable nor safe since, as you said:
> Some compilers may add less runtime checks and will create specialized code -> less safe.
There are several misconceptions about how people use Common Lisp.
> Users can portably and safely use assertions if they want that behavior
The point is that SBCL can check both at compile-time and at runtime. Which the others (exception: CMUCL and Scieneer CL) can't.
* (defun foo (a) (declare (string a)) (+ a 10))
; in: DEFUN FOO
; (+ A 10)
;
; caught WARNING:
; Derived type of A is
; (VALUES STRING &OPTIONAL),
; conflicting with its asserted type
; NUMBER.
; See also:
; The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
; caught 1 WARNING condition
FOO
That means SBCL tells me already at compile-time that there is a problem in my code. That's something which a development environment can exploit. I get a list of compiler warnings and can then fix my code - without the need to run it and to go into a break - and without the need to have a test case.
Now if I fix that error and run that code in my other implementation - that problem is also fixed there.
Now you assume that adding an assertion would have had a benefit for the other implementation, since it would then create a runtime assertion violation if not a string is provided.
The thing is: this is usually not done. Nobody writes code with assertions everywhere in Common Lisp.
What Lisp a developer really does: if we need to provide a runtime check, then we write a CLOS method.
Thus by default we encourage developers to use CLOS:
CL-USER 23 > (defmethod foo ((s string))
(concatenate 'string s "bar"))
#<STANDARD-METHOD FOO NIL (STRING) 4020259F8B>
CL-USER 24 > (foo 3)
Error: No applicable methods for #<STANDARD-GENERIC-FUNCTION FOO 406000173C> with args (3)
1 (continue) Call #<STANDARD-GENERIC-FUNCTION FOO 406000173C> again
2 (abort) Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
> Some compilers may add less runtime checks and will create specialized code -> less safe.
There is another misconception. Nobody forces me to use those compilers or use them in an unsafe mode.
The choice of compilers in Common Lisp is to give the user different tools for different situations. It does not force me to run my code containing type declarations in an unsafe mode - which usually is controlled by compiler switches: OPTIMIZE qualities for SAFETY, DEBUG and SPEED.
If I set safety to 3, most compilers will not create unsafe code - even though there are type declarations.
If we have a type violation in a CLOS method, SBCL even catches that:
* (defmethod baz ((s string)) (+ s 10))
; in: DEFMETHOD BAZ (STRING)
; (+ S 10)
;
; caught WARNING:
; Derived type of S is
; (VALUES STRING &OPTIONAL),
; conflicting with its asserted type
; NUMBER.
; See also:
; The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
; caught 1 WARNING condition
WARNING: Implicitly creating new generic function COMMON-LISP-USER::BAZ.
#<STANDARD-METHOD COMMON-LISP-USER::BAZ (STRING) {5132DEA1}>
The developer then will fix that code. When it then runs on another Lisp -> benefit.
So my advice usually is: even if one develops for another Lisp implementation, additionally use SBCL to check your code.
>Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB.
What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks.
Fear, uncertainty and doubt.
Type declarations are part of the ANSI Common Lisp standard.
Lisp implementations can decide what to do with type mismatches (thus the UB), but how can this be a problem? Code will still be standards-compliant.
The compiler tells you the usual (?) stuff like missing args, wrong named arguments, missing functions, undefined variables, syntax errors, etc.
But Common Lisp has also a (relatively primitive, compared to something like Haskell) type system and the SBCL compiler can make use of type declarations (for compile time type checking and for runtime type errors) and does also do some form of type inference.
With SBCL you get all the interactive dynamic features and a compiler which you can use to check your code and make it faster. The compiler can for example tell you which operations it could not optimize and why. Then you can decide if it is worth to put some effort into improving the code - for example by adding better type declarations. Common Lisp also lets you inline code or have some data be stack allocated...
Common Lisp also has a sophisticated error system, which allows you to get lots of excellent error reporting and handling.