葡京网上娱乐场Akka(34): Http:Unmarshalling,from Json

0.序

  先吐槽一下网上旧本子的Zeppelin和R的安,让自己折腾了几乎只钟头。

  不过最后或调通了也非便于,其实自己现一点R且没有学呢,只是刚看了一样节省课,但是是家伙既然出现在了Spark中,我思念她要生流行和好用的。

  之前安排了Zeppelin的Spark的构成,但是那没有配置R的片,但是Zeppelin既然能够起这般多之解释器,在我看来就是阳显他的一个别的软件所未具的优势:那便是于一个note里你得下多解释器来编排文字,生成图像,分析一个题材。

  再加上方便的导入导出和存储功能,我个人感觉未来风行只是岁月问题。

  废话尽量少说,进入正题:

 
Unmarshalling是Akka-http内把网上可传输格式的数变化成为程序高级结构话数据的经过,比如把Json数据易成有自定义类型的实例。按实际流程来说就是是先把Json转换成为可传输格式数据如果:MessageEntity,HttpRequest,HttpReponse等,然后重新更换成程序高级结构数据如classXX实例。Unmarshalling对一个A类实例到B类实例的更换是由此Unmarshaller[A,B]来促成的:

1.安装R及连锁的保:

只顾自己动用的是CentOS7的系,如果你无是自家之网可能下载的包会不相同。不过请不要操心,我会尽量说知道怎样找到相应系统的R的安装包。

首先走访官网https://www.r-project.org/

然后我们找到这个:

葡京网上娱乐场 1

接下来找到China那同样栏,随便点一个地方便哼。

下一场我们摘这:

葡京网上娱乐场 2

立下你应当能找到了咔嚓,里面含有几种植常用Linux发行版,我之凡CentOS,自然是选取RedHat系列了。

下一场我们依次点下去,直到点开README

葡京网上娱乐场 3

大好,这里像要安装一个EPEL的源来完成我们的R包的装置,这里我们进划横线的网址:

葡京网上娱乐场 4

箭头所倚凡这来自的RPM包,下载并设置的!

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -ivh epel-release-latest-7.noarch.rpm

接下自己打开这网址:https://zeppelin.apache.org/docs/0.6.2/interpreter/r.html

安相关R以及有关依赖:

yum install R R-devel libcurl-devel openssl-devel

安装时间比较长,耐心等待,去吆喝杯咖啡或看会别的,再回到吧!

但是你一旦是直密切看之口舌,它竟然在拍卖依赖关系之时段自动为安装上了open-jdk,没事,我们应用以下方法缓解:

wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.rpm 
rpm -ivh jdk-8u112-linux-x64.rpm 

重复于她装回去-。-醉了。

装有官网所说的依:

R -e “install.packages(‘devtools’, repos = ‘http://cran.rstudio.com’)”
R -e “install.packages(‘knitr’, repos = ‘http://cran.rstudio.com’)”
R -e “install.packages(‘ggplot2’, repos = ‘http://cran.rstudio.com’)”
R -e “install.packages(c(‘devtools’,’mplot’, ‘googleVis’), repos =
‘http://cran.rstudio.com’); require(devtools);
install_github(‘ramnathv/rCharts’)”

trait Unmarshaller[-A, B] extends akka.http.javadsl.unmarshalling.Unmarshaller[A, B] {...}
object Unmarshaller
  extends GenericUnmarshallers
  with PredefinedFromEntityUnmarshallers
  with PredefinedFromStringUnmarshallers {

  // format: OFF

  //#unmarshaller-creation
  /**
   * Creates an `Unmarshaller` from the given function.
   */
  def apply[A, B](f: ExecutionContext ⇒ A ⇒ Future[B]): Unmarshaller[A, B] =
    withMaterializer(ec => _ => f(ec))
...}

2.配置Zeppelin

复制官网的一样段落话:

从Unmarshaller的构建函数apply可以估计其的企图应该同函数A=>Future[B]充分相似。A代表网上可传输类型如MessageEntity、HttpRequest,B代表某种程序高级数据类型。因为A到B的变换是non-blocking的,所以可以马上回去Future类型结果。Akka-http按被移对象类型分类命名了下面这些类别别名:

Configuration

To run Zeppelin with the R Interpreter, the SPARK_HOME environment
variable must be set. The best way to do this is by
editing conf/zeppelin-env.sh. If it is not set, the R Interpreter will
not be able to interface with Spark.

You should also
copy conf/zeppelin-site.xml.template to conf/zeppelin-site.xml. That
will ensure that Zeppelin sees the R Interpreter the first time it
starts up.

个别个步骤:

  • 在conf/zeppelin-env.sh中加入SPARK_HOME这个环境变量
  •  cp conf/zeppelin-site.xml.template  conf/zeppelin-site.xml

此地不再赘述。

type FromEntityUnmarshaller[T] = Unmarshaller[HttpEntity, T]
type FromMessageUnmarshaller[T] = Unmarshaller[HttpMessage, T]
type FromResponseUnmarshaller[T] = Unmarshaller[HttpResponse, T]
type FromRequestUnmarshaller[T] = Unmarshaller[HttpRequest, T]
type FromByteStringUnmarshaller[T] = Unmarshaller[ByteString, T]
type FromStringUnmarshaller[T] = Unmarshaller[String, T]
type FromStrictFormFieldUnmarshaller[T] = Unmarshaller[StrictForm.Field, T]

3.重启Zeppelin:

/opt/zeppelin-0.6.2-bin-all/bin/zeppelin-daemon.sh restart

Akka-http对以下种类提供了活动的Unmarshalling转换: 

4.验证环境是否装好:

打开浏览器上Zeppelin的地点,然后notebook -> R Tutorial

如下图

葡京网上娱乐场 5

 

候编译完成,然后我截取了几乎摆设图片在脚:

葡京网上娱乐场 6

葡京网上娱乐场 7

葡京网上娱乐场 8

哼,我们的SparkR环境多建筑好了,Zeppelin又基本上矣一个新的工具,又精了众多。

PredefinedFromStringUnmarshallers
Byte
Short
Int
Long
Float
Double
Boolean
PredefinedFromEntityUnmarshallers
Array[Byte]
ByteString
Array[Char]
String
akka.http.scaladsl.model.FormData
GenericUnmarshallers
Unmarshaller[T, T] (identity unmarshaller)
Unmarshaller[Option[A], B], if an Unmarshaller[A, B] is available
Unmarshaller[A, Option[B]], if an Unmarshaller[A, B] is available

也就是说Akka-http提供了这些U类型的Unmarshaller[U,B]隐式实例。Akka-http也供了工具列Unmarshal:

object Unmarshal {
  def apply[T](value: T): Unmarshal[T] = new Unmarshal(value)
}

class Unmarshal[A](val value: A) {
  /**
   * Unmarshals the value to the given Type using the in-scope Unmarshaller.
   *
   * Uses the default materializer [[ExecutionContext]] if no implicit execution context is provided.
   * If you expect the marshalling to be heavy, it is suggested to provide a specialized context for those operations.
   */
  def to[B](implicit um: Unmarshaller[A, B], ec: ExecutionContext = null, mat: Materializer): Future[B] = {
    val context: ExecutionContext = if (ec == null) mat.executionContext else ec

    um(value)(context, mat)
  }
}

俺们好由此Unmarshal.to[B]把Unmarshal[A]转换成Future[B]。注意:这等同步只囊括了起网上可传输类型及程序类型转换这同进程,不包具体落实时的Json转换。下面是有Unmarshal的用例:

import akka.actor._
import akka.stream._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.http.scaladsl.model._
import akka.http.scaladsl.server.Directives._

object Unmarshalling {
  implicit val httpSys = ActorSystem("httpSystem")
  implicit val httpMat = ActorMaterializer()
  implicit val httpEC = httpSys.dispatcher

  val futInt = Unmarshal(43).to[Int]
  val futBoolean = Unmarshal("0").to[Boolean]
  val futString = Unmarshal(HttpEntity("Hello")).to[String]
  val futHello = Unmarshal(HttpRequest(method = HttpMethods.GET, entity = HttpEntity("hello")))

}

以上还是现已知道路中变换,可能没什么实际用,不像marshalling:中间层Marshalling有实在转换的要。Unmarshalling可以一直进行Json到从定义类型中的更换,如:

 val route = (path("User") & post) { entity(as[User]){ user =>
    complete(Future(s"inserting user: $user"))
  }} ~
    (path("Item"/IntNumber) & put) { id => entity(as[Item]){ item =>
      complete(Future(s"update item $id: $item"))
    }}

上述是由此Directive
as[???]实现的:

 /**
   * Returns the in-scope [[FromRequestUnmarshaller]] for the given type.
   *
   * @group marshalling
   */
  def as[T](implicit um: FromRequestUnmarshaller[T]) = um

眼看亟需将FromRequestUmarshaller[T]座落可视域内,FromRequestUmarshaller[T]实际是Unmarshaller[T,B]的别名: 

type FromRequestUnmarshaller[T] = Unmarshaller[HttpRequest, T]

以上篇讨论我们介绍了Akka-http的Marshalling是type-class模式之。其中重大得参照上篇讨论。现在咱们得这些Unmarshaller的隐式实例:

trait Formats extends SprayJsonSupport with DefaultJsonProtocol
object Converters extends Formats {
  case class User(id: Int, name: String)
  case class Item(id: Int, name: String, price: Double)
  implicit val itemFormat = jsonFormat3(Item.apply)
  implicit val userFormat = jsonFormat2(User.apply)
}

object Unmarshalling {
  import Converters._
...

假如下Json4s的贯彻方式,我们用如下提供这些隐式实例:

trait JsonCodec extends Json4sSupport {
  import org.json4s.DefaultFormats
  import org.json4s.ext.JodaTimeSerializers
  implicit val serilizer = jackson.Serialization
  implicit val formats = DefaultFormats ++ JodaTimeSerializers.all
}
object JsConverters extends JsonCodec

Json4s的现实用例如下:

  import scala.collection.mutable._
  case class User(id: Int, name: String)
  class Item(id: Int, name: String, price: Double)
  object AnyPic {
    val area = 10
    val title = "a picture"
    val data = ArrayBuffer[Byte](1,2,3)
  }

  val route = (path("User") & post) { entity(as[User]){ user =>
    complete(Future(s"inserting user: $user"))
  }} ~
    (path("Item"/IntNumber) & put) { id => entity(as[Item]){ item =>
      complete(Future(s"update item $id: $item"))
    }} ~
    (path("Picture") & put) { entity(as[AnyPic.type]){ pic =>
      complete(Future(s"insert picture: $pic"))
    }}

打效益上和表述灵活性来讲,Json4s的实现方式如占优。

下就此次讨论的示范源代码:

Unmarshalling

import akka.actor._
import akka.stream._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.http.scaladsl.model._
import akka.http.scaladsl.server.Directives._
import scala.concurrent._
import akka.http.scaladsl.marshallers.sprayjson._
import spray.json._

trait Formats extends SprayJsonSupport with DefaultJsonProtocol
object Converters extends Formats {
  case class User(id: Int, name: String)
  case class Item(id: Int, name: String, price: Double)
  implicit val itemFormat = jsonFormat3(Item.apply)
  implicit val userFormat = jsonFormat2(User.apply)
}

object Unmarshalling {
  import Converters._
  implicit val httpSys = ActorSystem("httpSystem")
  implicit val httpMat = ActorMaterializer()
  implicit val httpEC = httpSys.dispatcher

  val futInt = Unmarshal(43).to[Int]
  val futBoolean = Unmarshal("0").to[Boolean]
  val futString = Unmarshal(HttpEntity("Hello")).to[String]
  val futHello = Unmarshal(HttpRequest(method = HttpMethods.GET, entity = HttpEntity("hello")))

  val route = (path("User") & post) { entity(as[User]){ user =>
    complete(Future(s"inserting user: $user"))
  }} ~
    (path("Item"/IntNumber) & put) { id => entity(as[Item]){ item =>
      complete(Future(s"update item $id: $item"))
    }}

}

Json4sUnmarshalling

import akka.actor._
import akka.stream._
import akka.http.scaladsl.server.Directives._
import de.heikoseeberger.akkahttpjson4s.Json4sSupport
import org.json4s.jackson
import scala.concurrent._
trait JsonCodec extends Json4sSupport {
  import org.json4s.DefaultFormats
  import org.json4s.ext.JodaTimeSerializers
  implicit val serilizer = jackson.Serialization
  implicit val formats = DefaultFormats ++ JodaTimeSerializers.all
}
object JsConverters extends JsonCodec


object Json4sUnmarshalling {
  import JsConverters._
  implicit val httpSys = ActorSystem("httpSystem")
  implicit val httpMat = ActorMaterializer()
  implicit val httpEC = httpSys.dispatcher

  import scala.collection.mutable._
  case class User(id: Int, name: String)
  class Item(id: Int, name: String, price: Double)
  object AnyPic {
    val area = 10
    val title = "a picture"
    val data = ArrayBuffer[Byte](1,2,3)
  }

  val route = (path("User") & post) { entity(as[User]){ user =>
    complete(Future(s"inserting user: $user"))
  }} ~
    (path("Item"/IntNumber) & put) { id => entity(as[Item]){ item =>
      complete(Future(s"update item $id: $item"))
    }} ~
    (path("Picture") & put) { entity(as[AnyPic.type]){ pic =>
      complete(Future(s"insert picture: $pic"))
    }}
}