Seems we are witnesses of a new revolution in the corporate world caused by blockchain technologies. We’re moving quickly to virtual corporations, distributed autonomous corporations, smart contracts, new models of funding and so on. This article is about an exciting possibility of having distributed shareholders meetings where results are public and counted in trustless way.

Since late April I’m working on NXT cryptocurrency as a core developer. NXT is a 100% proof-of-stake cryptocurrency aiming “to transform a cryptocurrency into a decentralised financial platform that supports a thriving and fast-growing digital economy” (unlike hybrid Ripple’s approach which isn’t fully decentralized). Since May, 12th NXT has Asset Exchange (NXT AE) as a core feature. Anyone can issue assets on this totally decentralized and uncontrolled exchange and trade them. You can watch the assets being issued in web via several sites, e.g. NextBlocks, NXTReporting or SecureAE (you can even buy stocks with Bitcoins there). Spending some time on assets investigation you can find many possible applications of NXT AE, and many more are undiscovered yet.

In particular, you can fund your business via asset issuance getting money from anyone (not just angels/VCs) like KickStarter, but from day one and with no any fees. Probably you’ll get money from a community having a need for you product. It would be awesome to get feedback and community-powered decisions also, right?

With voting system I have finished recently that will be possible. The voting system will be introduced in NRS (NXT Reference Software) 1.3.x (x>0) or 1.4.0. The next stable release version is 1.2.1, so voting API & GUI will be accessible sometime in September, probably (earlier in testnet). But I can share some API details right now as they will be untouched probably.

Consider the example of an indy developer making a game. He got some funds via asset exchange. Then he made promising v.1 of the game. A concept is still raw as well as an implementation but fans are excited! So he wants to ask the community whether he should polish v.1 or make v.2 immediately built on top of better concept, or… So he’s starting the poll:

    val question = "Further directions in game development. "
    val description = "I got some great reviews. I have 10000 NXTs left. How should I spend them?"

    val finishBlockHeight = Nxt.getBlockchain.getHeight + 1440 // ~= 1 day
    val options = Array("Improve graphics and release it ASAP", "Start to work on v.2 to get funds for it's development", "Better ask experts, e.g. attend GameDev Conference", "Game is abortive. Stop working on it.")

    val optionModel = Poll.OPTION_MODEL_CHOICE
    val votingModel = Poll.VOTING_MODEL_ASSET

    val pb = new PollBuilder(question, desc, options, finishBlockHeight, optionModel, votingModel)
    val assetId: Long =  // assetId here
    pb.assetId(ai)
    pb.optionsNumRange(1, 1) // only 1 option to choose

    issueTxToGodId(new Attachment.MessagingPollCreation(pb), phrase1)

A vote could be sent with following code:

    val poll = Poll.getByName(question).head
    val vote = Array(0.toByte, 1.toByte, 0.toByte, 0.toByte)
    val attachment = new Attachment.MessagingVoteCasting(poll.getId, vote)
    issueTxToGodId(attachment, phrase2) 

1440 blocks after starting the poll we can get and print to console poll results(where 1 asset=1 vote):

    val pr = PollResults.get(poll.getId).get
    pr match {
        case cpr: nxt.PollResults#Choice =
            val m = cpr.getResults.toMap
            println(m)      
    }

So we have fair, cheap, public & distributed shareholders voting! Just imagine how it could change the world of business.

For calculating cosine similarity between two documents I used modified(and rewritten to Scala) example by Sujit Pal/Mark Butler(http://sujitpal.blogspot.ch/2011/10/computing-document-similarity-using.html, http://stackoverflow.com/questions/1844194/get-cosine-similarity-between-two-documents-in-lucene?rq=1). It’s working, but under big workload(hundreds of comparisons in second in my case) it consumes a lot of CPU / memory and even worse, some of threads getting stuck for few minutes(!) somewhere inside the Lucene code. So I had to rewrite that code to avoid RAMDirectory / IndexReader usage(I need no to store documents to Lucene storage at all). Here are two functions I have written and now want to share:

1. extractTerms function which extracts terms from document(presented as String) with stemming & stopwords removing:

    def extractTerms(content: String): Map[String, Int] = {    
        val analyzer = new StopAnalyzer(Version.LUCENE_46)
        val ts = new EnglishMinimalStemFilter(analyzer.tokenStream("c", content))
        val charTermAttribute = ts.addAttribute(classOf[CharTermAttribute])
   
        val m = scala.collection.mutable.Map[String, Int]()
   
        ts.reset()
        while (ts.incrementToken()) {
            val term = charTermAttribute.toString
            val newCount = m.get(term).map(_ + 1).getOrElse(1)
            m += term -> newCount       
        }
   
        m.toMap
    }  
   

2. similarity function which calculates similarity between two term vectors

    def similarity(t1: Map[String, Int], t2: Map[String, Int]): Double = {
        //word, t1 freq, t2 freq
        val m = scala.collection.mutable.HashMap[String, (Int, Int)]()
   
        val sum1 = t1.foldLeft(0d) {case (sum, (word, freq)) =>
            m += word ->(freq, 0)
            sum + freq
        }
   
        val sum2 = t2.foldLeft(0d) {case (sum, (word, freq)) =>
            m.get(word) match {
                case Some((freq1, _)) => m += word ->(freq1, freq)
                case None => m += word ->(0, freq)
            }
            sum + freq
        }
   
        val (p1, p2, p3) = m.foldLeft((0d, 0d, 0d)) {case ((s1, s2, s3), e) =>
            val fs = e._2
            val f1 = fs._1 / sum1
            val f2 = fs._2 / sum2
            (s1 + f1 * f2, s2 + f1 * f1, s3 + f2 * f2)
        }
   
        val cos = p1 / (Math.sqrt(p2) * Math.sqrt(p3))
        cos
    }
   

To calculate cosine similarity between text1 and text2 just call similarity(extractTerms(text1), extractTerms(text2))

In tests my code is 7-10 times faster and has less memory footprint. Enjoy!

Introduction

Probably you get some knowledge about cryptocurrencies from all that massive buzz around. But cryptocurrencies are not only buzz about another merchant accepting Bitcoin, market news, fun story involving dogecoin etc. A lot of tech possibilities we have now behind just money as digital property being transferring from one peer to another.

One of the notable alternative cryptocurrencies is Namecoin, which could be considered as currency and key-value database as well with auto-expiration of records injected in the blockchain. Owner of record has to pay (0.01 NMC for now) to have it in database for period of time of 36,000 blocks(~250 days).

You can insert any name-value record in the database, though there are some formal namespaces, two for now: DNS system for .bit domains(names starting with “d/”) and NameId which is fully decentralized OpenId alternative(names starting with “id/”).

Objects Counting

We are starting to get deeper into the Namecoin database with counting named objects in it.

First, install Namecoin daemon namecoind (very like Bitcoin’s bitcoind installation), for details, visit namecoin.info or namecoin.org . Wait while blockchain will be downloaded.

Then pull out all active records into out.txt file with

namecoind name_scan "" 1000000 > out.txt

To count all objects in database dump you have now use

cat out.txt | grep '"name"' | wc -l

To count .bit domains being registered

cat out.txt | grep '"name" : "d/' | wc -l

To count NameId entries

cat out.txt | grep '"name" : "id/' | wc -l

As of today, Feb 26th, 2014, Namecoin database contains 156276 entries in total, including 140237 .bit domains and 2919 NameIds.

Well, as scala.concurrent.Future class has no .flatten method, you could be wondered how to convert Future of Future e.g. Future[Future[T]] to just Future[T]. That’s can be easy done with combination of flatMap and identity functions:

//having fft:Future[Future[T]]
val ft:Future[Int] = fft.flatMap(x=> x) 

Play! Framework is probably the most popular Web framework for Scala Language(and one of the most popular Web frameworks for Java too). It’s stateless, it’s elegant, it’s a good friend of functional reactive programming.

To learn framework, the site contains very good tutorial, also there are some samples within ‘samples’ folder. And I’m happy to present another option to dive into the framework.

The sample application was developed with aim to test Mechanize Framework, but GistLabs published it as standalone application.

Application contains following examples:

• Controllers with different response codes: 302 redirect, 302 infinite redirect, 466 code, internal server error with custom message
• Cookies examples: printing cookies got in request, setting cookie
• XML examples: simple XML document output and echoing value passed in an input XML document passed with PUT request
• JSON example: echoing value passed in an input JSON document passed with PUT request
• Not-Modified example: on first request Etag/Cache-Control headers are sent in response, then Not-Modified result sent
• Forms: GET/POST forms, POST form with validation
• Files: single file upload, multiple files upload, POST/PUT/GET service example(create a file with random filename on POST, create a file with specified filename on PUT, then get file with GET request)
• Auth: login to see secret token, or try to access secret area directly(with redirect to index page)

Pull sources from GitHub and learn Play! by example!

P.S. Next part of the story will be about different Play! sample applications around the Web

What if you want to launch a part of your application within IDE to see log output? What if you want to have console launcher for actor system which is the part of your Play 2.x app? If your code uses Play classes(e.g. play.api.Logger) or it needs , you can’t write ordinary console app because it will throw “There is no started application” error during run.

How to get Play context within console application? Just extend your launcher class from play.core.StaticApplication class, pass path to application root to StaticApplication instance constructor. Simple example:

object ConsoleLauncher extends StaticApplication(new File(".")) {
    def main(args: Array[String]) {
        DataExtractionActorSystem.start()
    }
}

What if you want to remove dependency for Gradle task? For example, remove dependency from compileGroovy for compileJava(to add dependency in opposite dirrection and avoid cyclic dependency problem). Should be simple, but I didn’t find ready solution using search engines, though two minutes experiment provided me working code:

compileGroovy.taskDependencies.values -= "compileJava"

Ready solution to compile first Groovy classes then Java:

compileGroovy.taskDependencies.values -= "compileJava"
compileJava.dependsOn(compileGroovy)

I just released my first open-source component, Scala Client for BTC-e Trade and Public Data APIs! BTC-e.com is broker for Bitcoin/Litecoin/Namecoin/other cryprocurrencies trading. This post is about some choices made during development and how to use the clients.

Usage Details and Examples

• Implement ClientCredentials trait to connect to the Trade API :

    object MyCredentials extends ClientCredentials {
        val Key = "my key"
        val Secret = "my secret"
    }
  

• Initialize Trade API client as

    val tradeClient = new DefaultTradeApiClient(MyCredentials)
  

• Get free funds info with

    client.getInfo.map(println(_)) 
  

  It will print something like

    USD: 4.7 RUR: 2399 EUR: 0.0 BTC: 10 LTC: 19.99 NMC: 0.0 NVC: 0.0 TRC: 0.0 PPC: 0.0
  

• Cancel all open orders with following code

    tradeClient.orderList.getOrElse(List()).foreach{order=>
        tradeClient.cancelOrder(order.orderId)
    } 
  

• Create order to sell 200 litecoins for $4.99 each

    tradeClient.trade(Currency.LTC, Currency.USD, Direction.Sell, 4.99, 200.00)
  

• Close connections at the end

    tradeClient.releaseConnections 
  

• Initialize Public Data API as

    val pubClient = new DefaultMarketDataApiClient
  

• Get last deal price from ticker data for BTC/USD and print it:

    pubClient.ticker(Currency.BTC, Currency.USD).map{td=>
        println(td.last)
    }
  

See MarketDataApiClient and TradeApiClient classes for more functions.

Implementation and Customization Details

1. Common functions located in btce.scala, Trade API client in btce-trade.scala, Public Data API client in btce-marketdata.scala, Specs2 tests in BtceSpec.scala

2. There are many HTTP layer implementations. I implemented http requests/responses using WS framework from PlayFramework 2(Scala wrapper for Ning framework). If your project doesn’t use PlayFramework and/or already uses another HTTP framework(e.g. Apache HttpClient), make own implementation of HttpApiClient trait. Override functions getRequest(url: String): String (simple get request, it’s used by Public Data API), signedPostRequest(url: String, key: String, Secret: String, postBody: String): String (post request with already signed postBody, used by Trade API ), releaseConnections (shutdown connections pool here, if needed). Then define own Trade API client with code like class MyTradeApiClient(credentials: ClientCredentials) extends TradeApiClient(credentials) with MyHttpApiClient, class MyMarketDataApiClient extends MarketDataApiClient with MyHttpApiClient for Public Data client.

3. Enumerations chosen over sealed case classes hierarchy, e.g.

    object Direction extends Enumeration {
        type Direction = Value
        val Sell = Value("sell")
        val Buy = Value("buy")
    }
   

   It could be not the best choice in case of having in mind to build trading DSL over it. But I have no plans for trading DSL now.

4. No logging implemented in the released version to avoid extra dependency. If you incorporate a client into your software, add logging where needed(catch clauses, None results) with a logging framework project uses.

Again, the URL is https://github.com/kushti/btce-scala

Do you plan to change this world with a web startup? Thinking about technology stack? Monsterous Spring+hundreds of other Java frameworks or elegant, trendy but bit controversial Ruby on Rails? Don’t think about any compromises, think about Scala+PlayFramework 2!

What gives you Scala and PlayFramework combination?

• Play’s CLI(command line interface), hit refresh workflow, conciseness of Scala code and powerful abstractions provided by the framework(and dependent frameworks too, e.g. Specs2) give you stunning speed of development. In fact, you can have development speed typical for dynamic language while having all benefits of strong static typing. A startup needs for fast prototyping, so get it!

• A startup should be scalable to handle fast growth of userbase. Stateless framework architecture & built-in Akka support give you highest level of scalability.

• Scala is the JVM language means you can easily use thousands of opensource frameworks for map-reduce data processing, NLP, ML, genetic algorithms etc… Java was(and is) standard for academic open-source frameworks, #1 language for Apache Software Foundation(more than 100 opensource projects) etc. It adds speed to the prototyping, make your system more simple(one platform means less headache), also makes you development process much cheaper.

• Type safety gives you more stable and predictable development process(easier refactorings, avoiding of some types of errors etc). Unit tests are not enough, there is no doubt.

• Built-in asynchronous HTTP support makes modern web applications development easy.

P.S. I’m passionate about PlayFramework 2.x(already used it for 3 Scala and 1 Java projects). Next month I’m thinking about navigation plugin development(a bit like play navigator, but with formal FSM approach). Please write me if you want to contribute.

Trying to run Play2 + Morphia application, you can get such an error java.lang.IllegalArgumentException: can't parse argument number interface com.google.code.morphia.annotations.Id = @com.google.code.morphia.annotations.Id().

How to avoid it:

• Add SLF4JExtension for Morphia : http://code.google.com/p/morphia/wiki/SLF4JExtension. Here is the example how to add SBT dependencies, but please mind difference beetwen com.google.code.morphia and com.github.jmkgreen.morphia (make appropriate changes): https://github.com/leodagdag/play2-morphia-plugin/blob/master/project/Build.scala .

• Add 2 lines to init of your Global object(or beforeStart method)

    import com.google.code.morphia.logging.MorphiaLoggerFactory
    import com.google.code.morphia.logging.slf4j.SLF4JLogrImplFactory
    import play.api.GlobalSettings
  
    object Global extends GlobalSettings{
        MorphiaLoggerFactory.reset()
        MorphiaLoggerFactory.registerLogger(classOf[SLF4JLogrImplFactory])
    }