I have a question for those that have done a lot of programming in Lisp: Is macro programming a really big part of Lisp programming? I mean, what percentage of a typical project would be macros?
I ask the question because you can achieve the same results in most dynamic languages by grabbing the string representation of a function, run it through a parser, modify the resulting AST and then regenerating the function string before passing to an eval (CoffeeScript being a good example of this approach). Sure, it's messier than in Lisp, but in return for the messiness of macros, you get much easier to read syntax in most of your code. Is the use of macros really such a big part of Lisp programs that it justifies the lack of syntax everywhere else?
Macros are not something you need everyday (at least not your own macros) but when you need them you really need them. CoffeScript is a good example, if JavaScript would have been a real lisp there would be no need for CoffeeScript at all because you could build it in JS.
Your example of meta programming is very very messy and putting eval everywhere would also be a security nightmare and if you do it this way you don't end up with nice syntax.
<-- People don't really use this because of those reasons.
Macros are not messy to use they look like a normal function that meens often you don't even know or care if something is implmented in a macro or a function.
The other big part of the macros look like language build ins for example in a REST-Framework you would have something like 'defresource'. The auther of the framework can give you the best syntax to discribe what his library does. (Look on Stackoverflow for DSLs in Clojure)
Writting good macros your self is a little harder and can look wired because your just not used to code looking like this.
The USE of macros is a REALLY REALLY big part of every lisp programm because the HOLE language is build on macros. Almost everything that is normally build in the compiler is just a macro.
Firstly, CoffeeScript is built in JavaScript. Does that make it a 'real' Lisp?
Secondly, Lisp macros are every bit as big a security risk as using eval... That is what the E in REPL stands for, after all. In both cases, if you are taking input from source code, and not from outside sources, there is no security risk.
Thirdly, yes, my example is messy. But it is still doable, and I just have a hard time believing that this kind of programming is a major percentage of the code of a Lisp program. To be clear on that comment, I can easily believe that you end up invoking macros a lot, but I don't believe that a large percentage of the code written for a project is actually macros. What percentage of code in a typical project is macros? And how many of those macros can be easily reproduced by using specialised syntax of a language such as ruby (optional parantheses, blocks, re-opening classes, method_missing, etc)?
This is an important point, because if you only have a few hundred lines of this stuff in a typical Lisp program, and it can be reproduced by using a parser in another (dynamic) language, then the advantage just isn't that big. For argument's sake, imagine having a module available in Javascript that provides the Jison parser along with the Javascript grammar, which just re-emits the original Javascript source. In addition, it has a nice API to allow you to modify the grammar (adding / modifying rules). So far so easy, this stuff already exists in CoffeeScript, go copy it from there. Now, having loaded such a module, is Lisp really in a much better position than this modified Javascript?
In any language you can build a interpreter/compiler for any other language but thats not the point. The point is that you can do random AST transformation at compiletime.
You think of macros as a kind of pay of for s-exp syntax. I have to say that even without macros I would think s-exp syntax is nicer. Sure in Ruby you can do alot when you play around with that suger stuff but this stuff often doesn't come free. See this blogpost for an example of what I mean: http://briancarper.net/blog/579/keyword-arguments-ruby-cloju...
Depending on what you do you are writting a lot of macros. In lisp the interface to a library is almost always in a DSL style witch makes them easy to use. See this list here: http://stackoverflow.com/questions/3968055/are-there-any-clo.... In your own Application your maybe not going to need them that often depending on what you do.
What you discribe is just a macrosystem. Having macros in a language with syntax is duable (look at Dylan or Plot) but you will find that getting everything nice and easy to use is much harder then you think. If CoffeeScript has a macrosystem great. I don't really know CoffeeScript well enought. Could you provide me with some nice examples of how people use this in CoffeeScript?
If I where you I would just learn enought lisp that you can trie it out yourself.
How would you implement a new control structure that way? I think it won't be possible unless you write a parser for the whole language.
Take clojures "or" for example, it's just a macro. The language has only very few primitives [1] and that's the language, everything else is built out of that.
I ask the question because you can achieve the same results in most dynamic languages by grabbing the string representation of a function, run it through a parser, modify the resulting AST and then regenerating the function string before passing to an eval (CoffeeScript being a good example of this approach). Sure, it's messier than in Lisp, but in return for the messiness of macros, you get much easier to read syntax in most of your code. Is the use of macros really such a big part of Lisp programs that it justifies the lack of syntax everywhere else?