Data dodania Pytanie
2017-07-31 08:07 How to limit the range of prediction in Spark? »
I make a prediction in a dataset with stars range 1-5 using linear regression in Spark ml: val lr = new LinearRegression() .setMaxIter(10) .setRe...
(2) odpowiedzi
2017-07-30 18:07 How to stop Option-ality infecting everything »
I'm new to Scala. I'd like to have a class which expresses something like coordinates measured in positive integers. (This is a simplified example; pl...
(2) odpowiedzi
2017-07-30 16:07 Scala - Difference between map and flatMap »
Can anyone teach me property use cases of map and flatMap? In Option case, I know these two methods have each signature, def map(A => B): Option[B...
(2) odpowiedzi
2017-07-30 08:07 Spark - Appending multiple rows to create columns for a common column id »
I have a dataframe which currently looks like this |col_id|r_id_1|r_id_2|r_id_3| | 1 | a1 | b1 | c1 | | 1 | a2 | b2 | c2 | | 2 |...
(1) odpowiedzi
2017-07-30 07:07 Scala IntelliJ warning "dynamic invocation could be replaced with a constructor invocation" »
Why does Intellij give this warning, what does it mean, how can I make it better ? import akka.actor.Props object Router { def props(config: Confi...
(1) odpowiedzi
2017-07-29 16:07 Spark caching difference between 2.0.2 and 2.1.1 »
I'm a bit confused with spark caching behavior. I want to compute dependent dataset (b), cache it and unpersist source dataset(a) - here is my code: ...
(1) odpowiedzi
2017-07-29 11:07 Expression of type doesn't conform to Expected type B »
I have following code to understand covariant and lower bound, I intentionally make the code have compiling errors. The getOrElse works, which is sim...
(2) odpowiedzi
2017-07-29 08:07 Explode multiple columns in Spark SQL table »
There was a question regarding this issue here: Explode (transpose?) multiple columns in Spark SQL table Suppose that we have extra columns as below...
(2) odpowiedzi
2017-07-28 20:07 How to define schema of streaming dataset dynamically to write to csv? »
I have a streaming dataset, reading from kafka and trying to write to CSV case class Event(map: Map[String,String]) def decodeEvent(arrByte: Array[By...
(1) odpowiedzi
2017-07-28 17:07 Spark does not find Scala specific methods »
The problem is that every job fails with the following exception: Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([...
(1) odpowiedzi
2017-07-28 10:07 Can destructuring cause bugs in future? »
Initialy we have a code like class Person(firstName: String, lastName: String) val (firstName, lastName) = person Somebody swaps fields class Per...
(1) odpowiedzi
2017-07-27 20:07 @tailrec why does this method not compile with 'contains a recursive call not in tail position'? »
@tailrec private def loop[V](key: String): V = { key match { case _ => loop(key) } } This method doesn't compile and complain...
(1) odpowiedzi
2017-07-27 09:07 Scala 2.12 combination of Future and Either (biased version) »
I have two methods and both returns for Future[Either[String, T]] and want to chain them together to get the job done. I am using Scala 2.12 Either is...
(1) odpowiedzi
2017-07-27 08:07 How to write Haskell-do-notation in Scala »
I wrote the following Haskell code using do-nation. And I'd like to convert it to Scala code main :: IO () main = do print $ func1 (Right 1) (Rig...
(2) odpowiedzi
2017-07-27 07:07 alternate way to proceed without list in scala »
I have a scala code like this def avgCalc(buffer: Iterable[Array[String]], list: Array[String]) = { val currentTimeStamp = list(1).toLong // lo...
(2) odpowiedzi
2017-07-26 23:07 Is there any way to generate a UUID in Java that is identical to that of the one generated in C#? »
I'm porting a C# script into Spark (Scala) and I'm running into an issue with UUID generation in Scala vs GUID generation in C#. Is there any way t...
(1) odpowiedzi
2017-07-26 14:07 Mix pure function with side effect in scala »
I have a simple api method: def addWishToBoard(wish: Wish, boardId: BoardId, userId: UserId): Option[Board] It adds wish to board, if boardId and u...
(3) odpowiedzi
2017-07-26 11:07 write generic function that calls generic functions in scala »
I'm using Spark Datasets to read in csv files. I wanted to make a polymorphic function to do this for a number of files. Here's the function: def loa...
(1) odpowiedzi
2017-07-26 07:07 Extract the word from String using Regex »
I have the following Input: abc_account2621_activity_20161116_20161117_030627_311999667.csv xyx_account2622_click_2016111606_20161116_221031_3117...
(4) odpowiedzi
2017-07-25 19:07 Scala lambda function not resolved when declares the type of parameter? »
when defining method in Scala, I found this def method1: Int => Int = (j: Int) => j // works def method2: Int => Int = j => j // works ...
(2) odpowiedzi
2017-07-25 18:07 Scala: Find max sum of consecutive negative numbers in a List of Double »
I have a List[Double] with the following values: {-1.2200000000000006, -1.3200000000000003, -1.0099999999999998, 22.22, 11.11, -31.310000000000002, ...
(4) odpowiedzi
2017-07-25 12:07 What is type and what is type constructor in scala »
I'm a bit confused about understanding of what type in scala means. In documents I read that List[Int] is a type and List is a type constructor. bu...
(2) odpowiedzi
2017-07-25 02:07 Scala How to Sort List of Objects List by Object field (Object.field)? »
I have the following type of data: case class TipoDeDato[T] (nombreCampo: String,valor: T) And in my exercise, I need to create the following struc...
(1) odpowiedzi
2017-07-24 22:07 Extracting first defined value(if any) from sequence of suppliers of optional values »
Given a sequence of Supplier<Option<T>> -- e.g., a list of method references -- what's the idiomatic way to get the first defined result, ...
(2) odpowiedzi
2017-07-24 15:07 SHA256 of data stream »
I am having a use case where i am downloading the file from URL using GET request. Is it possible to calculate SHA256 of file stream without saving to...
(3) odpowiedzi
2017-07-24 04:07 Functor Instance for Type Constructor with Two Parameters in Scala »
I have a class Foo with two parameters, and I am trying to write a Functor instance for Foo with the first parameter fixed, as follows: object Scratc...
(2) odpowiedzi
2017-07-23 22:07 How to read HDFS file from Scala code »
I am new to Scala and HDFS: I am just wondering I am able to read local file from Scala code but how to read from HDFS: import scala.io.source objec...
(1) odpowiedzi
2017-07-23 21:07 Implicit conversion not applying »
I was trying to workout the classic example of converting arbitrary values into their Json representation and having compile time errors in case conve...
(1) odpowiedzi
2017-07-23 20:07 Solving dynamic programming problems using functional programming »
After you get a basic idea, coding dynamic programming (DP) problems in imperative style is pretty straightforward, at least for simpler DP problems. ...
(2) odpowiedzi
2017-07-23 13:07 Upper bounds in Scala »
I am trying to set Upper Bounds for a variable. trait Container { type A <: Number def value: A } Then when I try to get a value for my objec...
(3) odpowiedzi
2017-07-22 16:07 Shorthand to Init a Scala Enumeration ValueSet with one Value »
So I have a some Scala Enumeration object: object KnotType extends Enumeration { type KnotType = Value val STOPPER, HITCH, ENDLOOP, MIDL...
(1) odpowiedzi
2017-07-22 15:07 union types in scala with subtyping: A|B <: A|B|C »
I would like to have a A|B type to be the subtype of A|B|C. Is that possible to encode in Scala ? If yes, how ? I was hoping that I can make implicit...
(2) odpowiedzi
2017-07-22 14:07 How to synchronize Intellij and sbt builds for a scala project »
I have an sbt project that I have imported into Intellij. Sometimes I build the project at the command line using sbt, and then when I need to debug I...
(1) odpowiedzi
2017-07-21 23:07 Combining varargs and higher order functions »
Trying to use varargs with higher order functions. But the below doesn't seem to type/compile. def f(n: Any*) = {...} //compiles def f(fun: Any* =&g...
(1) odpowiedzi
2017-07-21 16:07 Scala: What'd be the idiomatic way to test equally spaced frequencies in array? »
I have a Scala application and have the following use case. Given a numberOfDates: Int and an optimalFrequencyInDays: Int I need to find the frequency...
(2) odpowiedzi