I've been searching for awhile now for a dynamic language to use as our "standard" in-house "scripting" tool. We've too many perls, pythons, bashes, and more running amok on various boxes and projects across our enterprise. In the last two years, I've become quite enamored with Ruby -- both with Rails and down at my Linux command prompt, where it finally gave Perl the "don't have to go home but you can't code here" hint -- but I've missed some of the libraries I'd taken for granted in the Java world.
And, for what we do, integration with Java is highly desirable. In my mind (and for the feature set I desire) this points to two possibilities: JRuby and Groovy.
JRuby is very nice. It has my nice comfortable Ruby syntax, open classes, and more. But two things trouble me currently:
- Java integration isn't fully there. You can write Ruby classes that call Java classes, BUT, you can't develop classes in Ruby and then make use of them from Java as if they were "real" java classes. The JRuby guys are moving in this direction, but it's not there yet.
- While others in our group really like Ruby, a few, especially the more junior guys, are still getting their heads around some of the more interesting (read painful) nuances of our architecture and J2EE technology in general. If I forced Ruby upon them now, they'd probably walk (albeit missing a great opportunity).
So, being the good PHB^H^H^H CIO that I am, I began the due diligence process. What due diligence, you ask? When looking at technology that could become a significant part of our development environment, I, as a PHB, look for general indicators that give warm fuzzy feelings (while in reality, could mean nothing):
- Review of the architecture and design decisions made by the developers.
- Size, responsiveness and general attitude of the community, particularly on the mailing lists.
- Current buzz surround the technology on blogs, tech pubs, etc.
- Benchmarks comparing the technology to other competing options.
- Commercial uptake.
Groovy (and Grails, to a certain extent) scores well on 1, 2, and 3 above. I like the technology so far, and from what I understand of the overall design it is very well thought out. The community is very welcoming, and the lists have been extremely helpful. There is quite a bit of effort to market Groovy and Grails, and with the release of GinA, seems like there is a new article daily.
However, when I went after 4 and 5, I was surprised. Regarding 4: I can't find a SINGLE benchmark comparing Groovy to competing technologies on the JVM (JRuby, Beanshell, Jython) or dynamic languages in general (Ruby, Python, Perl, and so on). Nor can I find any benchmarks concerning how Grails stacks up to JRuby on Rails, Ruby on Rails, vanilla Hibernate/Spring (something Grails uses under the covers), etc.
Surely, a technology that's got 1, 2, and 3 down has had someone do this sort of research?!? Surely someone close to the respective projects has asked these hard questions of themselves?
Because I know the previous paragraph has to, has to, be true, then I can only assume one thing: someone has done this as an exercise...and the results were depressing. Why else would the results not be published? If Groovy or Grails was faster, even slightly so, then you can bet it would trumpeted far and wide. The fact that it's not leaves an unsettling feeling.
I've asked the questions on the mailing lists, and I've only gotten vague wishful answers, to the tune of "well, we use Java (Spring/Hibernate) under the covers, so we've got to be as fast or faster". Really? Then where are the numbers? Why this gap?
I think it would behoove the Groovy guys (and Grails guys as well) to put some of this research in the public eye. I don't necessarily care that it's slower...I just want to know how it compares. It doesn't rule it out in any way regarding whether we use it or not...it only lets us know where we stand. It gives warm management fuzzies.
So I'm now working on my own benchmarks, but given time limitations it'll be rough. Don't hold it against me...after all, I'm just a PHB. I will definitely post the results here when I have them. Anyone with suggestions on the best way to approach this sort of benchmark, please let me know.
Regarding commercial uptake, or number 5 above, the explanations that are offered regarding this in both camps is that many big companies are using Groovy and Grails, but because of competitive advantage they won't allow themselves to be revealed. That's a hard point to prove or disprove, but it would help if there were at least a few successful products out there constructed in Groovy or Grails. In the Ruby on Rails world, it seems there are more than a few companies very willing to divulge that they use Rails...no doubt to ride the Rails marketing machine's coat tails. Why should we accept that, given significant adoption, the adverse would be true for Groovy/Grails?
So, there you have it. It is not my intent to offend anyone in the Groovy or Grails communities...as I said, you've been great so far. I'm only trying to give you the perspective of one decision maker, in one company, who is taking a hard look at your projects and assessing the viability of adoption. I really like what you guys are doing, but I need your help in understanding how well it's working. I'm sure others do as well.
9 comments:
The benchmark could be very interesting! :)
Regarding performance, there is a big distinction between Groovy and Grails. Languages and frameworks are different - I wouldn't combine the two. For one, Grails is awesome but it is still at 0.4.2 . Groovy is at 1.0 .
Also, if you peruse the Groovy mail lists, you'll find the common, successful strategy that's been applied regarding performance is to code up everything in Groovy and then check performance. Where there is a bottleneck or a concern, users have written that part in Java to pick up the performance.
IMO, not as a PHB but as project lead, benchmarks are not as important as a risk mitigation strategy. If the benchmark doesn't hit my application sweetspot - which I won't know until I actually build the app and *measure* the performance (not guess ahead of time) - it's worse than worthless for my project team - it directs focus (and wastes project team time) on the wrong thing. Let's say you are comparing Ruby vs Groovy. I would suggest that as a PHB, the risk mitigation strategy of dropping into C versus Java and how your staff can implement that strategy should be an area of greater focus in a decision for Ruby vs Groovy than benchmarks. Just because one is faster doing X doesn't mean that X won't still be a problem, let alone that Y, which you didn't benchmark, is the real problem.
Regarding commercial success - if you peruse the mail list again, it's not hard to figure out, even for a PHB :), which banks, insurance companies, beverage companies, etc... are using Groovy on real projects. If you're serious about Groovy and review the lists, there's plenty of material out there to meet any counter the concern about point 5.
Scott,
Good points, thanks.
Regarding coding up first and worrying later, it'd be really, really nice to have that sort of data beforehand. If there are certain things Groovy is just too slow to do, then it'd be nice to know that before we go down that path. I mean, seriously, on small projects I can see worrying downstream, but on big projects where significant investment is to be made in the actual coding, I don't think it's feasible to say "oh well, it may blow up when we put it in front of our targeted 4000 users, but we'll worry about that *after* we invest 3 months of development. My fear is that instead of one or two hot areas we'd have to dip into Java for optimization, there'll be 100. Code first, optimize later is fine for tried and true languages, but for something as new as Groovy I'm not sure it kind be explained away so easily.
Regarding your Ruby versus Groovy comment...ok, dropping into Java is indeed more appealing than dropping into C, but then what about Groovy compared to JRuby, Scala, etc?
Thanks,
John
A slight clarification on dropping into Java versus C - I'm not saying one is intrinsically better. It really depends on the comfort level of an organization's staff. IMHO, Groovy makes little sense for a department of C/C++ programmers. Likewise, Ruby makes little sense if you're a Java shop. Even if you don't like Groovy, there are something like 200 other language implementations that run on the JVM.
We may have to agree to disagree on the value of benchmarks. This may be a PHB versus tech lead difference of opinion but I think *especially* on large projects - you can't reliably predict where your bottlenecks will be and I think you're missing one of *the* fundamental advantages of using Groovy or Ruby: they are extremely expressive and you don't have a huge investment in the code itself.
Realistically, you don't have 100 places where performance is a problem - you have a few. If performance is that bad, you'll find out as soon developers start coding and writing unit tests.
And believe me, I'm not trying to explain away anything. I work on huge Groovy project with 45-50,000 SLOC with a seriously tight performance SLA. Early on, trying to tune performance, I was sure I knew where the problems were. After finding out I was wrong, I brought in Groovy core committer to look at the app. After looking at how we using Groovy, he was sure he knew what the problems. Finally, we looked at each, admitted that we were wrong about what we were 'sure' about and actually measured the app. Two Java classes later in places that we never would have guessed, the app was hitting the SLA. After years of software development, you would think that a person would learn this lesson once and remember it. I guess a dose of humility is good now and then :)
If you're looking a language other than Java for the JVM, review your reasons for do so. Presumably, it's the expressiveness of another language that's driving the decision. Don't lose sight of that. As long as there is a viable strategy for mitigating concerns, pick the language that's best for dealing with problem domain at hand. You'll be reading the code long after you write the code if the app has any shelf life at all.
Finally, it's a management myth that programmers spend most of the their time coding. On all the projects I've been over the last 20 years, I spend most of my time figuring out what to code by getting feedback from the the users. If I can an app working quickly with Groovy/Ruby/Jython and it actually does what the user expects, the time it takes to re- implement functionality in something more performant will get lost in the noise what's happening on the project plan.
I don't think benchmarks are really of too much importance, especially when comparing groovy and Java. Groovy will be slower than Java. Grails will be slower than a straight Spring/Hibernate concoction. I also don't think that you will find much documenting that because it was/is no ones intent to create a faster language or a slicker framework.
Where the benchmarking should be done is in development time. Grails was created so you don't need to make a straight Spring app each time. Groovy was created so you don't have to write bean methods every time, makes things more dynamic, while still keeping optional type safety (which makes a lot of people happy).
I don't feel that many people will start comparing languages/frameworks in areas that were outside their initial intention.
Sharing my experience with Java-Ruby (Ruby-Java) integration:
I used 'RJb' RubyJavaBridge. It is a gem that enables one to instantiate java objects from ruby code (yet they run on their own javaVM). The instances but fully callable from ruby, and there are some nice things like a transparent java String to ruby String translation.
just my .02
cies breijs.
I hope this: http://docs.codehaus.org/display/GRAILS/Grails+vs+Rails+Benchmark
Addresses your concerns
Cheers
Graeme
I believe I disagree with "I think *especially* on large projects - you can't reliably predict where your bottlenecks will be and I think you're missing one of *the* fundamental advantages of using Groovy or Ruby: they are extremely expressive and you don't have a huge investment in the code itself."
You always have an investment in the code - it represents the binding of your problem domain into a solution, and thus having to replace it has a cost. I accept that for yours, that cost has been small compared with the Groovy agility.
In a sense, I am going to make better choices if I have hard numbers, even if incomplete. My prior for the performance will be reasonable.
In at least one major project, we were hampered by the fundemental characteristics of the implementation technology. If my ORM does not cache in a sensible way, or if my presentation layer has fundemental weaknesses, I need to know pretty early.
To be clear - Groovy may be perfectly fast enough for my tasks, but I have no gut feel yet for what it does well, and what kind of penalty I am paying.
Put another way, the plethora of C++ vs Java benchmarks out there made it possible for me to write a performance-intensive NIH grant with some confidence that a Java implementation would be able to do the job. I did not find out three quarters of the way through that it was a non-starter, whereas in a different task, I seriously considered ripping out the persistence layer a month before shipping.
Microbenchmarks mean very little much of the time, but they do contribute to that gut feel. Getting some idea of whether my performance penalty is a factor of 0.2, 2, or 20 for a variety of tasks will inform whether I start down this road. Even if Groovy is pretty agile, days I spent implementing could have been spent optimizing, or tuning another implementation.
Scott
I think there is a missing point here when comparing Rails/Grails, that is about the feedback loop, I think the main reason for adopting these frameworks. Personally, I found that Rails is much more agile than Grails, it requires a lot less server restart and.... well... Grails server restart is slow. Anyway it's great to do Java web development, because it is surely more and more agile than doing it with "raw" java tools!
It's obvious that Grails outperforms Rails when in production, but that is because the Java platform is the result of ten years of work in this area.
Regarding early optimizations in big projects, it's always a bad idea, if you have a chance to cut down the feedback loop, you can focus on the real bottlenecks. The main point here is that with Groovy/Grails you should spend *hours* (not days) implementing and then move to optimize only these parts that really require optimization and not the others.
Post a Comment