Scala 2.10 Where did we came from, and where are we going to? Daniel Capó Sobral Last revision: Jan 05, 2013
May 10, 2015
Scala 2.10Where did we came from, and where are we
going to?
Daniel Capó Sobral
Last revision: Jan 05, 2013
Revision HistoryJul 20, 2012: add IsSeqLike and SearchingAug 08, 2012: removed previous revision – it
didn’t make 2.10.0; added raw interpolator; added :warnings; renamed makro to macros; @static; scaladoc grouping and link to anything
Aug 09, 2012: fixed stuff on macros and reflect; fixed stuff with Try; added references to other presentations
Sep 20, 2012: rescue replaced with recoverWith
Jan 05, 2014: Incorporate feedback about macros; improve actors; add links to some docs;
Who am I?Daniel C. Sobral @ Stack Overflowdcsobral @ Twitter, Gmail/Gtalk, Skype,
SliderShareDaniel Sobral @ Facebook, Speaker Deck,
LinkedInhttp://www.linkedin.com/in/danielsobralhttp://www.slideshare.net/dcsobralhttps://speakerdeck.com/u/dcsobralhttp://dcsobral.blogspot.com
What do I know about Scala?Scala gold badge @ Stack OverflowSmall contributions to Scala:
code optimizationenhancementsbug fixesdocumentation (not that small)
Active member of the Scala community (mailing lists, irc)
Blog about Scala
IMPORTANT!Scala 2.10.0 has not been released!
At the time this presentation has been written... (July, 07, 2012)
The information that follows may be changed before 2.10.0 is released.
SummaryHistoryNear Future
VersioningRelease ProcessBug fixesEnhancementsSIPs (Scala Improvement Process)ReflectionActors and Akka
Distant Future
The past
Release History
• 2.9.2
2012
• 2.9.0
• 2.9.1
• 2.8.2
2011
• 2.8.0
• 2.8.1
2010
• 2.7.3
• 2.7.4
• 2.7.5
• 2.7.6
• 2.7.7
2009
• 2.7.1
• 2.7.2
2008
• 2.3.2
• 2.3.3
• 2.4.0
• 2.5.0
• 2.5.1
• 2.6.0
• 2.6.1
2007
• 2.0.0
• 2.1.0
• 2.1.1
• 2.1.2
• 2.1.3
• 2.1.5
• 2.1.6
• 2.1.7
• 2.1.8
• 2.2.0
• 2.3.0
• 2.3.1
2006
• 1.4.0
2005
• 1.0.0
• 1.1.0
• 1.1.0
• 1.1.1
• 1.2.0
• 1.3.0
2004
• 0.8.1
• 0.8.2
• 0.9.0
• 0.9.1
2003
Some important landmarks
2006• Scalac written in Scala
• Implicits• Traits e linearization
• Multilline Strings
2007• Extractors• Private/protected primary constructors
• Private[this]• Placeholder parameters
• Early Initialization• Abstract Type Constructors
• Existential Types• Lazy values• Structural Types
2008• Java Generics• Case Class Extractors
2010• New Collections Design
• Binary Compatibility of Minor Releases
2011• Parallel Collections
Not comprehensive, by any means!
The near future
Explaining Versioning
2
Epoch
.9
Major
.1
Minor
-1
Bug fix
Versioning – Scala 2.10.0Major releaseBinary incompatible with previous versionsLanguage changesLibrary changesNew deprecationsRemoval of deprecated features
ClassesMethodsLanguage syntax
Examples of New DeprecationsOctal numbers!
Double ending in dot
scala> 077<console>:1: warning: Treating numbers with a leading zero as octal is deprecated. 077 ^res10: Int = 63
scala> 1.<console>:1: warning: This lexical syntax is deprecated. From scala 2.11, a dot willonly be considered part of a number if it is immediately followed by a digit. 1. ^res11: Double = 1.0
Examples of removal of deprecated featuresLanguage syntax
Class
Method
Package
for ( val i <- 1 to 10) println(i)
scala.collection.mutable.CloneableCollection
scala.collection.immutable.Queue.+
scala.testing.SUnit
Improvements of the release processProcess becoming more and more automatedPackage for RPM, APT, Brew, etc
Automatically generated!Milestones used to help integrate libraries
and frameworksM4 June 12M5 soon!
Release Candidates to consolidate and fix bugs
Bug fixesMany fixes!New pattern matcher!Ticket priorization!New warnings:
Detecting bugs in the compiler itself!More third party contributions!
git + pull request == FTW...besides the natural improvements of the
codebase.
Closed Tickets102 451 490 622 896 963 1118 1133 1195 1201 1247 1336
1430 1431 1432 1439 1510 1785 1799 2089 2094 2171 2196 2308
2322 2337 2388 2405 2435 2442 2460 2764 2782 2807 3047 3048
3098 3240 3272 3326 3343 3371 3392 3481 3566 3569 3628 3702
3708 3718 3755 3758 3761 3770 3798 3853 3854 3880 3898 3929
3960 3999 4018 4019 4025 4027 4063 4070 4098 4110 4124 4134
4147 4171 4172 4176 4216 4262 4270 4271 4273 4319 4326 4336
4355 4372 4391 4398 4417 4419 4425 4430 4441 4461 4467 4469
4478 4482 4490 4494 4501 4510 4512 4515 4535 4540 4541 4542
4547 4553 4561 4568 4570 4573 4574 4578 4579 4584 4592 4593
4595 4599 4601 4627 4636 4642 4647 4651 4656 4658 4661 4692
4696 4697 4709 4712 4713 4716 4717 4723 4727 4731 4737 4740
4749 4753 4757 4758 4759 4761 4764 4766 4770 4777 4780 4792
Closed Tickets4794 4800 4802 4804 4807 4809 4811 4818 4823 4827 4828 4831
4833 4839 4842 4846 4851 4853 4857 4858 4859 4860 4861 4869
4871 4874 4875 4877 4879 4882 4891 4894 4898 4899 4909 4910
4911 4925 4928 4929 4933 4935 4938 4954 4957 4959 4961 4963
4970 4975 4976 4979 4981 4985 4987 4989 5005 5009 5012 5018
5020 5023 5026 5029 5032 5033 5034 5037 5040 5041 5053 5054
5056 5062 5063 5066 5071 5072 5077 5078 5080 5083 5084 5085
5093 5096 5097 5099 5104 5105 5108 5117 5119 5121 5125 5127
5135 5137 5147 5148 5152 5156 5162 5165 5167 5168 5175 5176
5178 5189 5199 5201 5205 5206 5210 5212 5215 5223 5226 5229
5230 5239 5245 5248 5256 5259 5266 5267 5272 5284 5287 5291
5293 5295 5300 5305 5313 5317 5318 5328 5334 5336 5341 5343
5344 5352 5354 5356 5358 5359 5373 5374 5375 5377 5382 5384
Closed Tickets (442 – July 2nd)5387 5399 5405 5406 5407 5426 5428 5429 5441 5444 5446 5451
5452 5453 5455 5471 5488 5489 5497 5500 5510 5514 5522 5528
5530 5532 5535 5537 5542 5543 5544 5545 5552 5553 5554 5564
5571 5572 5577 5578 5580 5589 5590 5591 5593 5599 5608 5609
5610 5612 5614 5617 5623 5626 5627 5629 5632 5640 5641 5644
5648 5652 5654 5655 5656 5663 5666 5667 5672 5676 5677 5680
5682 5683 5688 5689 5690 5693 5696 5702 5703 5704 5706 5707
5708 5713 5715 5720 5728 5729 5735 5738 5742 5760 5761 5763
5769 5777 5779 5796 5801 5804 5805 5809 5816 5821 5829 5839
5840 5843 5845 5846 5853 5857 5862 5867 5879 5880 5881 5899
5910 5912 5914 5932 5953 5966 5967 5968 5971 5986 etc?
EnhancementsNew collections
Mutable SortedSet/SortedMap (AVL-based)Mutable Concurrent Map (TrieMap)Parallel Mutable Concurrent Map (ParTrieMap)
Performance:Immutable SortedSet/SortedMap
New (private) RedBlackTree TreeIterator
PartialFunction applyOrElse
BitSetMurmurHash3 (collections, xml, case classes)
New stuffType Classes:
Hashing (there’s an Equiv already)IsTraversableOnce and IsTraversableLike
Try (Twitter’s alternative to Either)Package scala.util.hashingConfigurable Pools for parallel collections@static@unspecializedto[Collection]???
New compile parametersoverride object (-Yoverride-objects)bytecode version 49, 50 and 51 (-target:jvm-1.x-asm)
-optimize faster and more effective (ok, not new)-Dscala.timings=true shows what’s is taking so
long to compileSIP-18: -language:XXX, -featurevarious logsvarious macrovarious patmatetc
New REPL commandsThe :warnings command will turn on all
hidden warnings, such as deprecation (that one is going to my finger’s muscle memory...)
scala.util.hashing.Hashing
def hashingOf[T : Hashing](obj: T): Int = implicitly[Hashing[T]].hash(obj)
// A bad hashing function for Seqsimplicit val seqHashing = Hashing fromFunction ((_: Seq[_]).size)
IsTraversableOnce andIsTraversableLike
class FilterMapImpl[A, Repr] (val r: GenTraversableLike[A, Repr]) { final def filterMap[B, That] (f: A => Option[B]) (implicit cbf: CanBuildFrom[Repr, B, That]) : That = r.flatMap(f(_).toSeq)}
// How to write the implicit conversion???
Obstacles to writing the implicit conversion:Array and String are not GenTraversableString does not have a type parameter (A)Nor BitSet and other collectionsInference over view bounds is difficult
IsTraversableOnce and IsTraversableLike provides a solution
IsTraversableOnce andIsTraversableLike
IsTraversableOnce andIsTraversableLike
class FilterMapImpl[A, Repr] (val r: GenTraversableLike[A, Repr]) { final def filterMap[B, That] (f: A => Option[B]) (implicit cbf: CanBuildFrom[Repr, B, That]) : That = r.flatMap(f(_).toSeq)}
implicit def filterMap[Repr, A](r: Repr) (implicit fr: IsTraversableOnce[Repr]) : FilterMapImpl[fr.A,Repr] = new FilterMapImpl(fr.conversion(r))
scala.util.Try
def percentCompleted(total: Int, done: Int): Int = Try ( done * 100 / total ) getOrElse 100
scala.util.TryTry { new FileInputStream(a)} recoverWith { case _: FileNotFoundException => Try(new FileInputStream(b))} recover { case _: FileNotFoundException => defaultInputStream} andThen { stream => in = stream.read(); stream.close(); in}
@staticTurn object fields into static fields of the object’s companion class
Fields are initialized through the static initializer of the class
Access to these fields are turned into direct static field access
Configurable Pools on Parallel Collectionsimport scala.collection.parallel._val pc = mutable.ParArray(1, 2, 3)
pc.tasksupport = new ForkJoinTaskSupport( new scala.concurrent.ForkJoinPool(2))
pc map { _ + 1)
pc.tasksupport = new ThreadPoolTaskSupport()
Configurable Pools on Parallel Collections – Customizingclass customTaskSupport extends TaskSupport { def execute[R, Tp] (task: Task[R, Tp]): () => R = ???
def executeAndWaitResult[R, Tp] (task: Task[R, Tp]): R = ???
def parallelismLevel: Int = ???}
to[Collection]scala> List(2, 1, 3).to[Seq]res17: Seq[Int] = Vector(2, 1, 3)
scala> List(2, 1, 3).to[Vector]res18: Vector[Int] = Vector(2, 1, 3)
scala> List(2, 1, 3).to[collection.mutable.Seq]res19: scala.collection.mutable.Seq[Int] = ArrayBuffer(2, 1, 3)
scala> List(2, 1, 3).to[collection.immutable.SortedSet]res20: scala.collection.immutable.SortedSet[Int] = TreeSet(1, 2, 3)
?????? it‘s a Nothing-returning that throws
exceptionsBecause it’s type is Nothing, ??? may replace
any expressionExcellent for stubs, exercises and
presentations!
???trait Opt [A] { // Exercise by Tony Morris def fold[X](some: A => X, none: => X): X = ??? def map[B](f: A => B): Opt[B] = ??? def get: A = ??? def flatMap[B](f: A => Opt[B]): Opt[B] = ??? def mapAgain[B](f: A => B): Opt[B] = ??? def getOrElse(e: => A): A = ??? def filter(p: A => Boolean): Optional[A] = ??? def exists(p: A => Boolean): Boolean = ??? def forall(p: A => Boolean): Boolean = ??? def foreach(f: A => Unit): Unit = ??? def isDefined: Boolean = ??? def orElse(o: => Opt[A]): Opt[A] = ??? // etc
Adapted from Tony Morris, http://blog.tmorris.net/further-understanding-scalaoption/
???trait Opt [A] { // Exercise by Tony Morris def fold[X](some: A => X, none: => X): X = ??? def map[B](f: A => B): Opt[B] = ??? def get: A = ??? def flatMap[B](f: A => Opt[B]): Opt[B] = ??? def mapAgain[B](f: A => B): Opt[B] = ??? def getOrElse(e: => A): A = ??? def filter(p: A => Boolean): Optional[A] = ??? def exists(p: A => Boolean): Boolean = ??? def forall(p: A => Boolean): Boolean = ??? def foreach(f: A => Unit): Unit = ??? def isDefined: Boolean = ??? def orElse(o: => Opt[A]): Opt[A] = ??? // etc
Adapted from Tony Morris, http://blog.tmorris.net/further-understanding-scalaoption/
EnhancementsScaladoc
Implicits!Diagrams!Link to anything!Group things!
API docsSteady stream of improvements on the
documents themselvesSee also:
http://docs.scala-lang.org/
ScalaDoc & Implicits
scaladoc -implicits
Class Diagrams!
scaladoc -diagrams
Parallel Collections Docs!
Available at http://docs.scala-lang.org/
Scaladoc – Link to anything![[x.y.Z.m]] links to m on Z, defaulting to the
type Z if m is present on both type and objectZ! forces type and Z$ forces object, to
disambiguatem can be written with type signature, to
disambiguate[[m]] refers to m on the current scopeIf dots or hashes are required in the type
signature, escape them with \
Scaladoc – Group things!Scaladoc now has attributes that allow
members to be grouped, and the groups described and prioritized
Members of the same group are displayed together
Groups of lower priority are seen before groups of higher priorities
See: @group, @groupdesc, @groupname and @groupprio
SIP – Scala Improvement ProcessSIP-11: String InterpolationSIP-12: Uncluttering Scala’s syntax for control
structuresSIP-13: Implicit ClassesSIP-14: Futures and PromisesSIP-15: Value ClassesSIP-16: Self-cleaning MacrosSIP-17: Type DynamicSIP-18: Modularizing Language FeaturesSIP-19: Implicit Source Locations
SIP – Scala Improvement ProcessSIP-11: String Interpolation (accepted!)SIP-12: Uncluttering Scala’s syntax for control
structures (postponed)SIP-13: Implicit Classes (accepted!)SIP-14: Futures and Promises (accepted!)SIP-15: Value Classes (accepted!)SIP-16: Self-cleaning Macros (postponed)SIP-17: Type Dynamic (accepted!)SIP-18: Modularizing Language Features
(accepted!)SIP-19: Implicit Source Locations (not accepted)
SIP-11: String InterpolationLack of string interpolation were a source of
constant complainsEven though very few languages support them!
Odersky says the difference between ${x} and "+x+" is a single characterIf you ignore the shift-typing...Someone says " and + can be typed without shift on
a swiss keyboard True!
About-face:What if string interpolation was more than string
interpolation?
SIP-11: String Interpolationdef hello(name: String = "world"): String = "Hello, " + name + "! "
// Interpolation!def hello(name: String = "world"): String = s"Hello, $name! "
SIP-11: String Interpolationdef percentCompleted(total: Int, done: Int): String = Try ( done * 100 / total ) map { percent => f"$percent%2d% completed" } getOrElse "100% completed"
def percentCompleted(total: Int, done: Int): String = Try ( f"${ done * 100 / total }%2d% completed" ) getOrElse "100% completed"
SIP-11: String Interpolation:How does it work?s"Hello, $name!" // becomesStringContext("Hello, ", "!").s(name)
f"${ done * 100 / total }%2d% completed" // becomesStringContext("", "%2d% completed").f(done * 100 / total)
raw"String with \n in it"
SIP-11: String InterpolationMultiline literals work too
s"""Ok!""" Interpolation may be nested
s"${ s"$x" }"Backslash is not used as escape character
r"\d+/$month" // not an actual interpolator! StringContext("""\d+/""").r(month)
But both s and f interpolators add the escaping themselves! s"Hello, $name!\n" // newline normal
But the raw interpolator does not!How to add/modify behavior:
Enrich My Library on StringContext Shadowing of object StringContext Added interpolators don’t actually need to be methods!
SIP-11: String Interpolation// Enrich My Libraryclass RegexContext(sc: StringContext) { def r(args: Any*) = { sc.checkLengths(args: _*) val res = (args, sc.parts.tail).zipped map { (arg, part) => s"\\Q$arg\\E$part" } mkString "" (sc.parts.head + res).r }}
implicit def toRC(sc: StringContext) = new RegexContext(sc)
SIP-11: String Interpolation// Interpolation nestingclass RegexContext(sc: StringContext) { def r(args: Any*) = { sc.checkLengths(args: _*) s"${sc.parts.head}${ (args, sc.parts.tail).zipped map { (arg, part) => s"\\Q$arg\\E$part" } mkString "" }".r }}
implicit def toRC(sc: StringContext) = new RegexContext(sc)
SIP-11: String Interpolation// Shadowing of StringContext objectobject StringContext(parts: String*) { def r(args: Any*) = { require(parts.length == args.length + 1) val res = (args, parts.tail).zipped map { "\\Q" + _ + "\\E" + _ } mkString "" (parts.head + res).r }}
// One can change s and f interpolators!
SIP-11: String Interpolation// apply and unapplySeq
def hello(name: String = "world"): String = i"Hello, $name!"
def who(message: String): String = message match { case i"Hello, $name!" => name case _ => “no clue"}
who(hello("Daniel")) == "Daniel"
SIP-11: String Interpolation// apply and unapplySeqimplicit class SI(sc: StringContext) { object i { def apply(args: Any*): String = sc.parts.head + (args,sc.parts.tail).zipped.map(_+_).mkString def unapplySeq(s: String): Option[Seq[String]] = { val partsr = sc.parts map (p => s"\\Q$p\\E") val r = (partsr mkString "(.*)").r s match { case r(xs @ _*) => Some(xs) case _ => None } } }}
SIP-13: Implicit Classes“Enrich My Library” (formely “Pimp My
Library”) very popularBut with lots of cerimony...“Extension methods” becoming common in
other languages (even Java!)And with less cerimony at that!
Solution: implicit class
SIP-13: Implicit Classes// String interpolation with Implicit Classesimplicit class RegexContext(sc: StringContext) { def r(args: Any*) = { sc.checkLengths(args: _*) val res = (args, sc.parts.tail).zipped map { "\\Q" + _ + "\\E" + _ } mkString "" (sc.parts.head + res).r }}
SIP-14: Futures and PromisesNon-blocking asynchronous of operations is a
common taskEvidence: many libraries have a Future
implementationIncluding Scala Actors library!
Problem:Non-standard interfaceInterface tied to implementation
Solution:Powerful and flexible API implementing the
concepts of Future and Promise
SIP-14: Futures and Promises// TODO – Sorry! Examples from SIP:
import scala.concurrent._val f: Future[List[String]] = future { session.getRecentPosts}f onComplete { case Right(posts) => for (post <- posts) render(post) case Left(t) => render("An error has occured: " + t.getMessage)}
SIP-14: Futures and Promises// TODO – Sorry! Examples from SIP:
import scala.concurrent._val f: Future[List[String]] = future { session.getRecentPosts}f onFailure { case t => render("An error has occured: " + t.getMessage)} onSuccess { case posts => for (post <- posts) render(post)}
SIP-14: Futures and Promises// TODO – Sorry! Examples from SIP:
import scala.concurrent._def main(args: Array[String]) { val rateQuote = future { connection.getCurrentValue(USD) } val purchase = rateQuote map { quote => if (isProfitable(quote)) connection.buy(amount, quote) else throw new Exception("not profitable") } blocking(purchase, 0 ns)}
SIP-14: Futures and Promises// TODO – Sorry! Examples from SIP:
import scala.concurrent.{ future, promise }val p = promise[T]val f = p.futureval producer = future { val r = produceSomething() p success r continueDoingSomethingUnrelated()}val consumer = future { startDoingSomething() f onSuccess { case r => doSomethingWithResult() }}
SIP-15: Value ClassesCurse of referencesCurse of boxing:
case class Meter(n: Int) much more “expensive” than int
Curse of implicit classes:"abc“.r creates an unnecessary object just to
call new Regex("abc")We want a class that is not a reference!Solution: value classes.
SIP-15: Value Classes// String interpolation with implicit value class!implicit class RegexContext(sc: StringContext) extends AnyVal { def r(args: Any*) = { sc.checkLengths(args: _*) val res = (args, sc.parts.tail).zipped map { "\\Q" + _ + "\\E" + _ } mkString "" (sc.parts.head + res).r }}
SIP-15: Value ClassesCan have only one parameter
Which must be a public valFor instance, [T: Ordering](v: T) is not acceptableCan be extended by traits
If they are either Any or AnyValCannot define equality or hash code
But may be case classes!Equality and hash code are applied over the
parameterThey are effectively “final”On a local scope, the class is optimized away!If they escape scope, they are boxed
SIP-16: MacrosAcceptance postponed, but available
experimentally on Scala 2.10.0Large compilers are hard to maintain
Pressure against new features on the compilerCompiler Plugins can be used to extend Scala...
...but they are difficult to keep in sync with scalacPossible solution: Macros
“Macros? Ugh!” –C/C++ trauma!Something new: scala.reflection
Macros become almost free!
SIP-16: MacrosScala CATs!
Compile-time AST Transformations Abstract Syntax Tree
Inspired by Nemerle macrosFour kind looked into:
Typed def macros – This is the only made available!Untyped def macrosType (class/trait) macrosAnnotation macros
Official Overview:http
://docs.scala-lang.org/overviews/macros/typemacros.html
SIP-16: Macros// Typed def macros
(1 to 5).foreach(println) ^ ^ | + Type check + Macro execution + New type check
SIP-16: Macros// Untyped def macros
for {val i = 0; i < 10; i += 1} println(i)^ ^ | || + Arguments not type checked!+ Macro execution+ Type check
SIP-16: Macros// Untyped def macros – why?
for {val i = 0; i < 10; i += 1} println(i) ^ ^ | “i” not visible in this scope! + + “i” only visible in this scope.
SIP-16: Macros// Type (class/trait) macros
class User extends DbTable(jdbcUrl, “t_user”) ^ + Macro
// Connects to database at compile time// and gets table description to create// class “User”
SIP-16: MacrosType macros are similar to F#’s type
providers
Next in line of developmentFirst user likely to be Typesafe’s Slick
(formely ScalaQuery)http://slick.typesafe.com/
SIP-16: Macros// Macro annotations
@memoizedef fat(n: Int) = if (n > 0) n * fat(n – 1) else 1
// Macro applied over the annotated definition,// be it class, method, parameter, etc
SIP-16: MacrosOnly typed macros made available...And with experimental status, at that...Behind flag –language:experimental.macros...But already used in some places on the library!C/C++ macros trauma very widespread......but the true C++ macros are templates! (imho)Macro Paradise used to experiment with other
types of macros:http://docs.scala-lang.org/overviews/macros/paradise.html
SIP-16: MacrosTyped Macros Advantages:
Relatively simple to implementThere’s no difference in the type signature of a
method implemented with a macro! Easy to test
Hygienic macros provided through a macro! Therefore, self-cleaning
Non-hygienic Macros are also easy to create
SIP-16: MacrosDisadvantages:
Somewhat limitedTwo-stages compilation
It’s not possible to define and use a macro on the same compilation
Non-hygienic by defaultNay-sayers will go on nay-saying.
SIP-16: Macrosobject Trimmer { def trim(s: String): String = macro trimImpl
import scala.reflect.macros.Context def trimImpl(c: Context)(s: c.Expr[String]): c.Expr[String] = { import c.universe._ // see reflection
val Literal(Constant(untrimmed: String)) = s.tree val trimmed = untrimmed.lines.map(_.trim).mkString("\n")
c.Expr(Literal(Constant(trimmed))) }}
SIP-16: MacrosPath dependent method types:
def m(c: Context)(param: c.Expr[Any]): c.Expr[Any]
Through –Ydependent-method-types parameter on previous versions
Reflection, reflection and reflection! “Once we do reflection right, we get macros
for free!” – paraphrasing Martin Odersky
SIP-17: Type DynamicProblems:
Scala’s integration to other JVM languages assumes static typing Lots of dynamically typed languages on the JVM!
Static typing also hinders some flexible DSLsSolution: trait DynamicSimilar to “missing method”Makes it easy to integrate with other JVM
languagesHas great DSL potentialAnd other uses as well...
SIP-17: Type Dynamicscala> class xmlPath(xml: scala.xml.Node) extends Dynamic { | def selectDynamic(path: String) = xml \\ path | }defined class xmlPath
scala> new xmlPath(<root><a><b><c/></b></a></root>).bres9: scala.xml.NodeSeq = NodeSeq(<b><c/></b>)
SIP-17: Type DynamicMethod call conversions:
Standard methods:applyDynamic
Methods with named parameters:applyDynamicNamed
Setters:updateDynamic
Getters (nullary methods):selectDynamic
SIP-17: Type DynamicAlso in the works is DynamicProxy
Add/Change methods, have everything else be automatically redirected
Might (maybe) even do so with static typing, through macros!
However, not yet in the libraryMight make 2.10.0 or not – time will tell
Problems:Some Scala features are hard to understandOthers induce errorsAnd some are experimental
But they are all necessary, for one reason or another!
Solution: SIP-18
SIP-18: Modularizing Language Features
Controls access to some language featuresUsing these features cause compilation
warningsWhich might become compilation errors in the
futureAccess to features allowed through:
compile flagsimplicit in scope
Access can be allowed through inheritedScalaDoc explains why control and why make
available
SIP-18: Modularizing Language Features
Contentious features:Postfixed operatorsMethod invocation made using implicit reflectionImplicit conversionsHigher kinded typesExistentials
Except for those equivalent to Java wildcard typesType Dynamic
Experimental FeaturesMacros
SIP-18: Modularizing Language Features
SIP-18: Modularizing Language Features// Postfix operator
scala> "abc" lengthwarning: there were 1 feature warnings; re-run with -feature for detailsres0: Int = 3
SIP-18: Modularizing Language Features// Postfix operator, using -feature
scala> "abc" length<console>:8: warning: postfix operator length should be enabledby making the implicit value language.postfixOps visible.This can be achieved by adding the import clause 'import language.postfixOps'or by setting the compiler option -language:postfixOps.See the Scala docs for value scala.language.postfixOps for a discussionwhy the feature should be explicitly enabled. "abc" length ^res0: Int = 3
SIP-18: Modularizing Language Features// Methods called using reflection
scala> val x = new AnyRef { def hello = println("world") }x: Object{def hello: Unit} = $anon$1@7d628303
scala> x.hello<console>:10: warning: reflective access of structural type member method hello should be enabledby making the implicit value language.reflectiveCalls visible.This can be achieved by adding the import clause 'import language.reflectiveCalls'or by setting the compiler option -language:reflectiveCalls.See the Scala docs for value scala.language.reflectiveCalls for a discussionwhy the feature should be explicitly enabled. x.hello ^world
SIP-18: Modularizing Language Features// Implicit conversions
scala> implicit def f(s: String): Int = Predef.augmentString(s).toInt<console>:8: warning: implicit conversion method f should be enabledby making the implicit value language.implicitConversions visible.This can be achieved by adding the import clause 'import language.implicitConversions'or by setting the compiler option -language:implicitConversions.See the Scala docs for value scala.language.implicitConversions for a discussionwhy the feature should be explicitly enabled. implicit def f(s: String): Int = Predef.augmentString(s).toInt ^f: (s: String)Int
The feature being limited are implicit conversions
Other implicit usages are still allowedImplicit conversions through implicit classes
works too
SIP-18: Modularizing Language Features
SIP-18: Modularizing Language Features// Higher Kinded Types
scala> class Monad[M[_]]<console>:9: warning: higher-kinded type should be enabledby making the implicit value language.higherKinds visible.This can be achieved by adding the import clause 'import language.higherKinds'or by setting the compiler option -language:higherKinds.See the Scala docs for value scala.language.higherKinds for a discussionwhy the feature should be explicitly enabled. class Monad[M[_]] ^defined class Monad
SIP-18: Modularizing Language Features// Existentials
scala> val l: List[ T forSome { type T }] = List(1)<console>:7: warning: the existential type T forSome { type T }, which cannot be expressed by wildcards, should be enabledby making the implicit value language.existentials visible.This can be achieved by adding the import clause 'import language.existentials'or by setting the compiler option -language:existentials.See the Scala docs for value scala.language.existentials for a discussionwhy the feature should be explicitly enabled. val l: List[ T forSome { type T }] = List(1) ^l: List[T forSome { type T }] = List(1)
SIP-18: Modularizing Language Features// Type Dynamic
TODO
SIP-18: Modularizing Language Features// Macros
scala> def g(d: Any): Any = macro f<console>:12: error: macro definition needs to be enabledby making the implicit value language.experimental.macros visible.This can be achieved by adding the import clause 'import language.experimental.macros'or by setting the compiler option -language:experimental.macros.See the Scala docs for value scala.language.experimental.macros for a discussionwhy the feature needs to be explicitly enabled. def g(d: Any): Any = macro f ^
scala.reflectAllows us to explore the structure of a program,and interact with that structure
Wikipedia: reflection happens at compile timeScala: you use the same API at compile time as
well, with macrosMain Concepts:
Reflection Libraries and UniversesNames, symbols, types and treesMirrors
Overview will be provided at:http://docs.scala-lang.org/overviews/reflection/overview.
html
scala.reflect – Universes
Source: Scala Reflection Pre-SIP
Notice: all of the classes on scala-library.jar depicted above were removed.
scala.reflect – UniversesContains a number of interrelated elements
which, together, make up the main components of reflectionSymbolTreeNameetc...
Use of path dependent types to relate types with the universes they came from
scala.reflect – UniversesBuilt in layers with different degrees of detail using
the “cake pattern” :Base:
Fundamental concepts Identity, e and little else
JavaUniverse Manipulation through JVM-provided reflection, enhanced
with information provided by ScalaMacros
Manipulation at compile time Compiler’s universe, but presented with a more restricted
APIOthers: compiler
scala.reflect – Main Types
Universe
Name
Symbol
Type
Tree
Position
AnnotationInfo
FlagSet
scala.reflect – Name
Name
TermName
TypeName
scala.reflect – NameScala has two namespaces:
TypesValues (Term)
Name ties a string to one of those namespacesName also enables convertion of names from
language-representation to class-file representation
scala.reflect – Symbol
Symbol
Type
Symbol Type ...
owner
Symbol
Name
TermName
TypeName
scala.reflect – SymbolAll definitions are tied to symbols
ClassesMethodsParametersVariablesIf you named it, it has a symbolIf you did not name it, but refer to it as being
“anonymous”, it also has a symbolImmutable at run time
At macro time, one can change its flags
scala.reflect – TypeStructure associated with a symbol:
Class membersParameters and return types of a methodetc
Made up of case classesHandled through pattern matching
scala.reflect – Types and Symbolsscala> import scala.reflect.runtime.universe._import scala.reflect.runtime.universe._
scala> typeOf[List[Int]]res0: reflect.runtime.universe.Type = List[Int]
scala> res0.member(newTermName("head"))res1: reflect.runtime.universe.Symbol = method head
scala> res1.typeSignatureres2: reflect.runtime.universe.Type = => A
scala> res1.typeSignatureIn(res0)res3: reflect.runtime.universe.Type = => Int
scala.reflect – Types and Symbolsdef intMethods[T : TypeTag](v: T) = { val IntType = typeOf[Int] val vType = typeOf[T] val methods = vType.members.collect { case m: MethodSymbol if m.isPublic => m -> m.typeSignatureIn(vType) } methods collect { // Pattern-matching various method types: case (m, mt @ NullaryMethodType(IntType)) => m -> mt case (m, mt @ MethodType(_, IntType)) => m -> mt case (m, mt @ PolyType(_, MethodType(_, IntType))) => m -> mt }}
scala.reflect – Types and Symbolsdef intMethods[T : TypeTag](v: T) = { val IntType = typeOf[Int] val vType = typeOf[T] val methods = vType.members.collect { case m: MethodSymbol if m.isPublic => m -> m.typeSignatureIn(vType) } methods collect { // Since m is a MethodSymbol, we can do it like this instead: case (m, mt) if m.returnType == IntType => m -> mt }}
scala.reflect – Types and Symbolsdef intMethods[T : TypeTag](v: T) = { val IntType = typeOf[Int] val vType = typeOf[T] val methods = vType.members.collect { case m: MethodSymbol if m.isPublic => m -> m.typeSignatureIn(vType) } methods collect { // Actually, you need this to account for type aliases: case (m, mt) if m.returnType =:= IntType => m -> mt }}
scala.reflect – Tree
Treechildren• Tree• Tree• ...• Tree
Type Position Symbol
TypTree TermTree
scala.reflect – TreeCompiler’s internal representation of source
codeImmutable at runtime, made up of case
classesHowever, at macro time one can change its
symbol, type and positionEssential when writing macrosAlso used for annotated typed
scala.reflect – Treescala> import scala.tools.reflect.ToolBoximport scala.tools.reflect.ToolBox scala> import scala.reflect.runtime.{currentMirror => m}import scala.reflect.runtime.{currentMirror=>m} scala> val tb = m.mkToolBox()tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = scala.tools.reflect.ToolBoxFactory$ToolBoxImpl@5bbdee69 scala> val tree = tb.parseExpr("1 to 3 map (_+1)")tree: tb.u.Tree = 1.to(3).map(((x$1) => x$1.$plus(1))) scala> val eval = tb.runExpr(tree)eval: Any = Vector(2, 3, 4)
scala.reflect – Treescala> import scala.reflect.runtime.universe._import scala.reflect.runtime.universe._
scala> showRaw(reify({ def f(x: Int, y: Int) = x + y }).tree)res4: String = Block(List(DefDef(Modifiers(), newTermName("f"), List(), List(List(ValDef(Modifiers(PARAM), newTermName("x"), Ident(scala.Int), EmptyTree), ValDef(Modifiers(PARAM), newTermName("y"), Ident(scala.Int), EmptyTree))), TypeTree(), Apply(Select(Ident(newTermName("x")), newTermName("$plus")), List(Ident(newTermName("y")))))), Literal(Constant(())))
scala.reflect – Mirror
InstanceMirror
ClassMirror FieldMirror MethodMirror
ModuleMirror
ClassMirror
MethodMirror
(constructors)
scala.reflect – MirrorMirrors allow one to manipulate classes and
instances at run timeEach Universe may have one or mirrorsEach mirror is tied to a class loaderClasses with the same name may be loaded
by distinct class loadersMirrors allows reflection over the Scala
language, instead of its JVM implementation
scala.reflect – Mirrorval obj = "String"val objClass = obj.getClassval classClassLoader = objClass.getClassLoaderval classLoaderMirror = runtimeMirror(classClassLoader)val classSymbol = classLoaderMirror.classSymbol(objClass)val classType = classSymbol.typeSignatureval methodName = newTermName("length")val methodSymbol = classType.member(methodName).asMethodSymbolval instanceMirror = classLoaderMirror.reflect(obj)val methodMirror = instanceMirror.reflectMethod(methodSymbol)methodMirror.apply() // == (obj.length: Any)
Actors and AkkaThe standard actors library will be removed
Replaced by Akka’sAkka will be part of 2.10’s distribution
Only binaries – source code will remain apartThe old library will also be available, on a separate
jarThere is a migration guide:
http://docs.scala-lang.org/overviews/core/actors-migration-guide.html
The standard actors has a migration kitThis provides a layer over standard and Akka actors
to make them source code compatible with each other
The distant future
Wild Speculationsuntyped macros
for { val x = 0; x < 10; x += 1 } println(x)
macro typesclass X extends MacroClassY(parm)
macro annotations@memoize def fat(n: Int) = if (n > 0) n * fat(n -1) else 1
effect systemabstract types/type parameters unificationpre-processor (DOA!)
Other presentationsAlex Boisvert’s Scala Roadmap:
https://speakerdeck.com/u/boia01/p/scala-roadmap-as-of-april-2012
Goes into much more depth about “distant future”
Simon Ochsenreither’s What’s New in 2.10http://
ochsenreither.posterous.com/whats-new-in-210No explanations, but more comprehensive, and
links to commits