Data dodania Pytanie
2017-02-02 00:02 Deleting Rows from multiple tables in a slick query »
I googled and found this thread Slick 3.0: Delete rows from multiple tables in a transaction However the solution says that its not a good solution. ...
(2) odpowiedzi
2017-02-01 22:02 Specifying the filename when saving a DataFrame as a CSV »
Say I have a Spark DF that I want to save to disk a CSV file. In Spark 2.0.0+, one can convert DataFrame(DataSet[Rows]) as a DataFrameWriter and use ...
(1) odpowiedzi
2017-02-01 17:02 Unable to convert Scrooge generated classes to maps using Shapeless »
I'm using Scrooge to generate classes. They look something like this, here's an example: object Flags extends ThriftStructCodec3[Flags] { private v...
(0) odpowiedzi
2017-02-01 11:02 What is an entity in Akka-Http? »
I am new to akka-http and building a basic server-client application in scala. The examples I looked at has the object "entity". Can someone please ex...
(1) odpowiedzi
2017-01-31 18:01 Testing an array for emptiness in Scala »
The question isn't about how to test an array for emptiness (arr.length == 0 does this fine). Rather my question is, why does scala> Array().isE...
(2) odpowiedzi
2017-01-31 13:01 FTP does not work from mesos docker container »
I have scala application with akka steams. So the flow of my application is like this: 1. Check if file exists on FTP - I'm doing it with the org.apa...
(1) odpowiedzi
2017-01-31 09:01 Scala: type check on yield that may return an empty list »
I have to following code in Scala, that's meant to return sentences. val x: List[Word] = for (word <- words) yield { if (word.isLegit()) sentenc...
(2) odpowiedzi
2017-01-30 22:01 Efficient implementation of Catamorphisms in Scala »
For a datatype representing the natural numbers: sealed trait Nat case object Z extends Nat case class S(pred: Nat) extends Nat In Scala, here is a...
(2) odpowiedzi
2017-01-30 21:01 Scala pattern matching with grep »
I am learning scala from Odersky's book Programming in Scala: A Comprehensive Step-by-Step Guide This code object n3 { def main(args: Array[Strin...
(2) odpowiedzi
2017-01-30 12:01 Is connection pooling inherently done in cloudant database? »
I am making an application using akka-http with cloudant database. I have seen the documentation and there is max_connections variable, but I am still...
(1) odpowiedzi
2017-01-29 13:01 How to get the coefficients of the best logistic regression in a spark-ml CrossValidatorModel? »
I train a simple CrossValidatorModel using logistic regression and spark-ml pipelines. I can predict new data, but I'd like to go beyond the black box...
(1) odpowiedzi
2017-01-29 08:01 Dropping Columns by Data type in Scala Spark »
df1.printSchema() prints out the column names and the data type they possess. df1.drop($"colNames") will drop by column name. Is there a way to ada...
(1) odpowiedzi
2017-01-29 07:01 Why my Scala imperative style map creation snippet is slower than Java one? »
I'm new to Scala, I'm trying to create a big map from IndexedSeq, I found a mention on SO that functional style map creation is much slower than imper...
(2) odpowiedzi
2017-01-28 22:01 Scala: Add values to sortedSet or array using for loop »
For a specific application I need to add integers from a non-sorted array to an empty sortedSet using a for loop. (Or to a new array which is sorted)....
(1) odpowiedzi
2017-01-28 12:01 Deterimine concrete type without explicit matching »
I am trying to make this DRYer: type CollectPredicate = PartialFunction[Option[SqlArgument], SQLActionBuilder] val cp: CollectPredicate = { cas...
(1) odpowiedzi
2017-01-28 00:01 General Questions about Akka and Typesafety »
Question 1: The JVM does not know about generics, therefore type parameters in Scala (and Java) only exist at compile-time. They don't exist at run-t...
(1) odpowiedzi
2017-01-27 18:01 Understanding type Parameters in Scala »
I am trying to understand the type parameters in Scala. Let's look the following general example: def func1[T](a : T) : T = a I understand that fun...
(1) odpowiedzi
2017-01-27 17:01 Spark each file to a dataset row »
I have many files in a directory, each file containing text spanning multiple lines. Currently I use the following code to read all those files to a s...
(2) odpowiedzi
2017-01-27 11:01 Preserve results of sbt incremental compilation when moving project directory »
Consider the scenario: I have a simple scala project managed by sbt, in a directory called foo. cd foo sbt compile It now takes a few seconds and t...
(0) odpowiedzi
2017-01-26 20:01 Apply list of functions to value using Scala Cats »
in haskell I could do the following to a string let f = sequence [id, reverse] f "test" I am at a bit of a loss how to approach this in a better m...
(1) odpowiedzi
2017-01-26 13:01 SBT 0.13/1.0 - replacment for <<= / <+= operators to redefine tasks? »
Currently SBT tells me that <<= I'm using to add a trigger certain events is deprecated. updateBrowsers <<= updateBrowsers.triggeredBy(fa...
(1) odpowiedzi
2017-01-26 07:01 Scala how to get last calculated value of stream? »
According to scala docs stream implements lazy lists where elements are only evaluated when they are needed. Example; val fibs: Stream[BigInt] = BigI...
(1) odpowiedzi
2017-01-25 19:01 Scala abstract class,what does self do? »
I am learning about abstract classes,this link is very informative scala generic method overriding abstract class Foo[T] { self:T => def bar1(...
(1) odpowiedzi
2017-01-25 16:01 Dropping empty DataFrame partitions in Apache Spark »
I try to repartition a DataFrame according to a columnm the the DataFrame has N (let say N=3) different values in the partition-column x, e.g: val my...
(0) odpowiedzi
2017-01-25 15:01 Spark rdd.count() yields inconsistent results »
I'm a bit baffled. A simple rdd.count() gives different results when run multiple times. Here is the code i run: val inputRdd = sc.newAPIHadoopRDD(...
(0) odpowiedzi
2017-01-24 14:01 Spark DataFrame - drop null values from column »
Given a dataframe : val df = sc.parallelize(Seq(("foo", ArrayBuffer(null,"bar",null)), ("bar", ArrayBuffer("one","two",null)))).toDF("key", "valu...
(2) odpowiedzi
2017-01-24 12:01 Scala anonymous function syntax and return type »
I have found few kinds of anonymous function syntax in scala: val m5_1 = { (n: Int) => n * 5 } val m5_2 = (n: Int) => { n * 5 } : Int val m5_...
(3) odpowiedzi
2017-01-24 04:01 Derived classes as class constructor parameters »
[EDITED BELOW] I have a class hierarchy that is, right now, roughly as follows: sealed trait Foo { def method: Any } case class Bar extends Foo ...
(2) odpowiedzi
2017-01-23 20:01 Why do implicit value classes have an extra method invocation? »
I was inspecting the byte-code generated by implicit classes and wanted to compare to what is generated when they extend AnyVal. Without implicit: o...
(1) odpowiedzi
2017-01-23 17:01 Why can you not flatMap a Set when the elements are wildcarded but you can flatMap a Seq? »
Why won't the Scala compiler let you flatMap a Set[Set[_ <: SomeType]] but will let you flatMap a Seq[Seq[_ <: SomeType]]? This behaviour is dem...
(1) odpowiedzi
2017-01-23 13:01 Passing a map with struct-type key into a Spark UDF »
I want to write an Spark 1.6 UDF which takes the following map: case class MyRow(mapping: Map[(Int, Int), Double]) val data = Seq( MyRow(Map((1, 1...
(1) odpowiedzi
2017-01-23 01:01 Select column by name with multiple aggregate columns after pivot with Spark Scala »
I am trying to aggregate multitple columns after a pivot in Scala Spark 2.0.1: scala> val df = List((1, 2, 3, None), (1, 3, 4, Some(1))).toDF("a",...
(1) odpowiedzi
2017-01-22 03:01 Value Class w/ String Inner Type? »
Given the following function: def prefixDr(firstName: String, lastName: String): String = "Dr. " + firstName + " " + lastName Let's say I deci...
(2) odpowiedzi
2017-01-22 01:01 Call-by-name in Scala vs lazy evaluation in Haskell? »
Haskell's lazy evaluation will never take more evaluation steps than the eager evaluation. On the other hand, Scala's call-by-name evaluation may req...
(3) odpowiedzi
2017-01-22 01:01 Predef.StringCanBuildFrom's behaviour »
I stumbled accross Predef.StringCanBuildFrom surprising implementation that breaks assumptions I was making on CanBuildFrom in my code. Here's the imp...
(1) odpowiedzi