CI Weekly #23 | 0 起源开启持续集成的顶级姿势(内涵福利)

58到家|微服务架构的两大解耦利器与顶级实践

本文谈论了58
到家的微服务架构,以及微服务两大解耦利器配置基本和信息总线的特级实践,希望带给你有的参阅。(via:
雪夜凋零

 

不停交付的实践与琢磨

“人管代码,代码管机器
,其实不止交付就能达到那个功能”。那篇小说分析了干吗要有不止交付、持续交付能一呵而就怎么着、持续交付的原则、实践分析等等。(via:
不一会之学

咱俩看看服务端对request
entity的操作是以ByteString举行的。客户端上传一串字符的request如下:

本周二晚,fir.im 的后端工程师纪承在 SF
大讲堂开启了一场关于持续集成的技能直播,分别从管理者、团队开发人员等不同角度,演讲咋样渐进式、最小成本地力促不断集成的过程,点击这里可围观重播视频,配合演讲
slide

一起食用最佳 🙂

 

flow.ci

 

The Must Know Checklist For DevOps & Site Reliability Engineers

这篇著作为
IT行业的不比地点分享了一部分必要的技巧和实用的资源,如DevOps,Site
Reliability Engineers,IT Architects,Data Scientists
..etc,值得参考。(via : Aymen El
Amri


以上是 CI Weekly #23的有着技能分享,
如有问题,请联系我们~

Happy building!
flow.ci

CI Weekly 围绕『 软件工程效能提升』
举办一层层技术内容分享,包括国内外持续集成、持续交付,持续部署、自动化测试、
DevOps 等实施学科、工具与资源,以及部分工程师文化有关的程序员 Tips
。同步于
flow.ci
Blog、微信公众号、法定今日头条和讯专栏简书,欢迎关注或投稿:)

客户端上传数据范例:


import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpEntity.limitableByteSource
import akka.http.scaladsl.model._

import scala.concurrent.duration._
import akka.util.ByteString

import scala.concurrent._
import scala.util._

object ClientEntity extends App {

  implicit val sys = ActorSystem("ClientSys")
  implicit val mat = ActorMaterializer()
  implicit val ec = sys.dispatcher

  def runService(request: HttpRequest, rentity: RequestEntity) = {
   val futResp = for {
     entity <- Future.successful(rentity)
     resp <- Http(sys).singleRequest(
       request.copy(entity = rentity)
     )
   } yield resp

   futResp
    .andThen {
      case Success(r@HttpResponse(StatusCodes.OK, _, entity, _)) =>
        val futEnt = entity.dataBytes.map(_.utf8String).runForeach(println)
             Await.result(futEnt, Duration.Inf) // throws if binding fails
             println("End of stream!!!")
      case Success(r@HttpResponse(code, _, _, _)) =>
        println(s"Download request failed, response code: $code")
        r.discardEntityBytes()
      case Success(_) => println("Unable to download rows!")
      case Failure(err) => println(s"Download failed: ${err.getMessage}")

    }
  }

  val reqText = HttpRequest(uri = s"http://localhost:8011/text")
//  runService(reqText,HttpEntity.Empty)
//    .andThen{case _ => sys.terminate()}

  val postText = HttpRequest(HttpMethods.POST,uri = s"http://localhost:8011/text")
  val uploadText = HttpEntity(
    ContentTypes.`text/plain(UTF-8)`,
    // transform each number to a chunk of bytes
    ByteString("hello world again")
  )
//  runService(postText,uploadText)
//    .andThen{case _ => sys.terminate()}

  val reqRandom = HttpRequest(uri = s"http://localhost:8011/random")
 //   runService(reqRandom,HttpEntity.Empty)
 //    .andThen{case _ => sys.terminate()}

  val numbers = Source.fromIterator(() =>
    Iterator.continually(Random.nextInt()))
    .map(n => ByteString(s"$n\n"))
  //make conform to withoutSizeLimit constrain
  val source = limitableByteSource(numbers)

  val bytes = HttpEntity(
    ContentTypes.`application/octet-stream`,
    // transform each number to a chunk of bytes
    source.take(10000)
  )
  val postRandom = HttpRequest(HttpMethods.POST,uri = s"http://localhost:8011/random")
  runService(postRandom,bytes)
    .andThen{case _ => sys.terminate()}


}

flow.ci

用下面的法子调用:

DevOps实践-打造自服务持续交付

这篇著作讲了 DevOps
转型的出生政策和推行,技术上摘取了一种轻量级、低耦合的构成-Ansible+Jenkins+AWS,值得借鉴。(via:
ThoughtWorks

 

Jenkins & Docker 持续集成实践

该散文出自于ADDOPS团队,是一篇有关Jenkins 与 Docker
的穿梭集成实践,介绍了Jenkins
实现持续集成,并帮忙Docker插件做CD的执行案例,希望能带给你有些启迪!(via:公众号
HULK一线技术随笔 )

 

再来看看本期 CI Weekly ,整理了有赞、58到家的微服务实践、Docker
持续集成、DevOps实践和工具链分享等,一起来探视~

  def runService(request: HttpRequest, rentity: RequestEntity) = {
   val futResp = for {
     entity <- Future.successful(rentity)
     resp <- Http(sys).singleRequest(
       request.copy(entity = rentity)
     )
   } yield resp

   futResp
    .andThen {
      case Success(r@HttpResponse(StatusCodes.OK, _, entity, _)) =>
        entity.dataBytes.map(_.utf8String).runForeach(println)
      case Success(r@HttpResponse(code, _, _, _)) =>
        println(s"Download request failed, response code: $code")
        r.discardEntityBytes()
      case Success(_) => println("Unable to download rows!")
      case Failure(err) => println(s"Download failed: ${err.getMessage}")

    }
  }

有赞支付微服务实践

2015年到2016里边,有赞业务流量增长,架构格局遭受了挑战,集团起头通往业务拆分和服务化方向迈进;二〇一七年工作需要,又多方向微服务架构渐变,中间也遇上题目,比如架构升级与实践。(via:
有赞技术团队)

  val reqRandom = HttpRequest(uri = s"http://localhost:8011/random")
    runService(reqRandom,HttpEntity.Empty)
     .andThen{case _ => sys.terminate()}

最近,flow.ci
联合 Coding 推出了一拨儿促销活动,如若您碰巧是 Coding
黄金/钻石会员,这一次福利要吸引啦:9 月 1 日前可在 Coding x flow.ci
活动页面
可领取
2 个月的 flow.ci
基础版套餐兑换券,具体的应用表明请看这里,有任何疑窦可发送「在线信息」联系
flow.ci 客服~

 

只顾:ContentType只是一种备注,不影响线上多少表明形式,线上的数目永远是ByteString。可是,其中的application/octet-stream类型代表数量必须是Source[ByteString,Any]。我们下边就通过客户端的例证来精晓HttpEntity。上边是一个客户端测试函数:

import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
import akka.http.scaladsl.Http
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.model._
import akka.util.ByteString
import akka.http.scaladsl.model.HttpEntity._

import scala.util.Random

object ServerEntity extends App {

  implicit val httpSys = ActorSystem("httpSystem")
  implicit val httpMat = ActorMaterializer()
  implicit val httpEC = httpSys.dispatcher


  val numbers = Source.fromIterator(() =>
    Iterator.continually(Random.nextInt()))
    .map(n => ByteString(s"$n\n"))
  //make conform to withoutSizeLimit constrain
  val source = limitableByteSource(numbers)



  val route =
    path("random") {
      get {
        withoutSizeLimit {
          complete(
            HttpEntity(
              ContentTypes.`application/octet-stream`,
              // transform each number to a chunk of bytes
              source.take(1000))
          )
        }
      } ~
        post {
          withoutSizeLimit {
            extractDataBytes { bytes =>
              val data = bytes.runFold(ByteString())(_ ++ _)
              onComplete(data) { t =>
                complete(t)
              }
            }
          }
        }
    } ~ path("text") {
      get {
        complete("Hello World!")
      } ~
        post {
          withoutSizeLimit {
            extractDataBytes { bytes =>
              val data = bytes.runFold(ByteString())(_ ++ _)
              onComplete(data) { t =>
                complete(t)
              }
            }
          }
        }
    }


  val (port, host) = (8011,"localhost")

  val bindingFuture = Http().bindAndHandle(route,host,port)

  println(s"Server running at $host $port. Press any key to exit ...")

  scala.io.StdIn.readLine()

  bindingFuture.flatMap(_.unbind())
    .onComplete(_ => httpSys.terminate())

}

虽然complete(“Hello
World!”)有些迷糊,可是相应complete做了些字符串到ByteString的变换。我们得以从下边这么些run瑟维斯(Service)(Service)函数得到印证。下面是以此例子的调用:

很明确,HttpEntity可以分两大类,一种是Strict类型的,它的data是ByteString。另一种是UniversalEntity类型,它的数码dataBytes是Source[ByteString,Any]。无论如何末了在线上的如故ByteString。HttpEntity的ContentType注明了传输数据格式,有:

 

   futResp
    .andThen {
      case Success(r@HttpResponse(StatusCodes.OK, _, entity, _)) =>
        val futEnt = entity.dataBytes.map(_.utf8String).runForeach(println)
             Await.result(futEnt, Duration.Inf) // throws if binding fails
             println("End of stream!!!")
      case Success(r@HttpResponse(code, _, _, _)) =>
        println(s"Download request failed, response code: $code")
        r.discardEntityBytes()
      case Success(_) => println("Unable to download rows!")
      case Failure(err) => println(s"Download failed: ${err.getMessage}")

    }
    } ~ path("text") {
      get {
        complete("Hello World!")
      } ~
        post {
          withoutSizeLimit {
            extractDataBytes { bytes =>
              val data = bytes.runFold(ByteString())(_ ++ _)
              onComplete(data) { t =>
                complete(t)
              }
            }
          }
        }
object ContentTypes {
  val `application/json` = ContentType(MediaTypes.`application/json`)
  val `application/octet-stream` = ContentType(MediaTypes.`application/octet-stream`)
  val `text/plain(UTF-8)` = MediaTypes.`text/plain` withCharset HttpCharsets.`UTF-8`
  val `text/html(UTF-8)` = MediaTypes.`text/html` withCharset HttpCharsets.`UTF-8`
  val `text/xml(UTF-8)` = MediaTypes.`text/xml` withCharset HttpCharsets.`UTF-8`
  val `text/csv(UTF-8)` = MediaTypes.`text/csv` withCharset HttpCharsets.`UTF-8`

  // used for explicitly suppressing the rendering of Content-Type headers on requests and responses
  val NoContentType = ContentType(MediaTypes.NoMediaType)
}

客户端:

  val numbers = Source.fromIterator(() =>
    Iterator.continually(Random.nextInt()))
    .map(n => ByteString(s"$n\n"))
  //make conform to withoutSizeLimit constrain
  val source = limitableByteSource(numbers)

 

  path("random") {
      get {
        complete(
          HttpEntity(
            ContentTypes.`application/octet-stream`,
            // transform each number to a chunk of bytes
            source.take(10000)
          )
        )
      } ~
object HttpEntity {
  implicit def apply(string: String): HttpEntity.Strict = apply(ContentTypes.`text/plain(UTF-8)`, string)
  implicit def apply(bytes: Array[Byte]): HttpEntity.Strict = apply(ContentTypes.`application/octet-stream`, bytes)
  implicit def apply(data: ByteString): HttpEntity.Strict = apply(ContentTypes.`application/octet-stream`, data)
  def apply(contentType: ContentType.NonBinary, string: String): HttpEntity.Strict =
    if (string.isEmpty) empty(contentType) else apply(contentType, ByteString(string.getBytes(contentType.charset.nioCharset)))
  def apply(contentType: ContentType, bytes: Array[Byte]): HttpEntity.Strict =
    if (bytes.length == 0) empty(contentType) else apply(contentType, ByteString(bytes))
  def apply(contentType: ContentType, data: ByteString): HttpEntity.Strict =
    if (data.isEmpty) empty(contentType) else HttpEntity.Strict(contentType, data)

  def apply(contentType: ContentType, contentLength: Long, data: Source[ByteString, Any]): UniversalEntity =
    if (contentLength == 0) empty(contentType) else HttpEntity.Default(contentType, contentLength, data)
  def apply(contentType: ContentType, data: Source[ByteString, Any]): HttpEntity.Chunked =
    HttpEntity.Chunked.fromData(contentType, data)
...

 

从显示的结果可以得出run瑟维斯(Service)函数中的entity.dataBytes.map(_.utf8String)已经把ByteString转换成了String,也就是说服务器端发送的Entity里的多少是ByteString。

再示范一下在客户端用Source上传数据。服务端代码:

 
大家说过Akka-http是一个好的序列融为一体工具,集成是通过数据交流形式实现的。Http是个在网上传输和吸纳的正儿八经协商。所以,在动用Akka-http此前,可能我们依旧需要把Http形式的网上数据沟通细节明白了解。数据交流双方是通过Http消息类型Request和Response来促成的。在Akka-http中对应的是HttpRequest和HttpResponse。这多少个品类都有所HttpEntity类型来装载需要沟通的数码。首先,无论咋样数据在线上的表现格局肯定是一串bytes。所以,数据交流六头Request,Response中的Entity也非得是以bytes来发挥的。在Akka-http里我们把需要传输的多寡转换成ByteString,通过网络发送給接收端、接收端再把收到消息Entity中的ByteString转换成目的项目标数量。这三个转移过程就是Akka-http的马尔斯halling和Unmarshalling过程了。我们先从HttpEntity的构建函数来询问它的定义:

俺们领悟Akka-http是遵照Akka-Stream的,具备Reactive-Stream效能特色。下边我们就示范一下怎么举办stream的上传下载。首先定制一个Source:

 

  val reqText = HttpRequest(uri = s"http://localhost:8011/text")
  runService(reqText,HttpEntity.Empty)
    .andThen{case _ => sys.terminate()}

咱俩只需要对这几个函数传入RequestEntity就可以了然重回Response里Entity的无数细节了。首先大家要求服务端发送一个纯字符串Hello
World。服务端代码如下:

 

 } ~ path("text") {
      get {
        complete("Hello World!")
      } ~

 

服务端也是用HttpEntity来装载这些Source然后经过HttpRequest传给客户端的:

从地点商讨我们询问了在马尔斯hal,Unmarshal下层只是ByteString的操作和转换。上边是此次商量示范源代码:

服务端:

       post {
          withoutSizeLimit {
            extractDataBytes { bytes =>
              val data = bytes.runFold(ByteString())(_ ++ _)
              onComplete(data) { t =>
                complete(t)
              }
            }
          }
        }

我们在客户端或者用runService(Service)来分析传过来的entity。由于接到一个巨型的Source,所以需要修改一下收受形式代码:

  val postText = HttpRequest(HttpMethods.POST,uri = s"http://localhost:8011/text")
  val uploadText = HttpEntity(
    ContentTypes.`text/plain(UTF-8)`,
    // transform each number to a chunk of bytes
    ByteString("hello world again")
  )
  runService(postText,uploadText)
    .andThen{case _ => sys.terminate()}

大家可以见到放进entity里的数目是ByteString。

 

 

 

咱俩再试着发送一些数据給服务端,然后让服务端把结果通过response
entity再次来到来:

 

 val numbers = Source.fromIterator(() =>
    Iterator.continually(Random.nextInt()))
    .map(n => ByteString(s"$n\n"))
  //make conform to withoutSizeLimit constrain
  val source = limitableByteSource(numbers)

  val bytes = HttpEntity(
    ContentTypes.`application/octet-stream`,
    // transform each number to a chunk of bytes
    source.take(10000)
  )
  val postRandom = HttpRequest(HttpMethods.POST,uri = s"http://localhost:8011/random")
  runService(postRandom,bytes)
    .andThen{case _ => sys.terminate()}