java

How to register the Java SPark UDF in spark shell?

烈酒焚心 提交于 2021-02-19 07:35:10
问题 Below is my java udf code, package com.udf; import org.apache.spark.sql.api.java.UDF1; public class SparkUDF implements UDF1<String, String> { @Override public String call(String arg) throws Exception { if (validateString(arg)) return arg; return "INVALID"; } public static boolean validateString(String arg) { if (arg == null | arg.length() != 11) return false; else return true; } } I am building the Jar with this class as SparkUdf-1.0-SNAPSHOT.jar I am having a table name as sample in hive

Count Elements in a stream and return Integer insted of long

浪子不回头ぞ 提交于 2021-02-19 07:35:08
问题 I need to count Elements in a Stream and assign it to an Integer without casting. .count() does return long thought about the .collect(Collectors.reducing(..)) but cant figure it out. I feel like there is something simple I don't get. My Try: Stream<String> s = Stream.of("Hallo ", "Test", "String"); Integer count = s.filter(e -> (e.length() >= lb && e.length() <= ub && !e.contains(" "))) .map(e -> e.toUpperCase()) .distinct() .collect(Collectors.reducing(0, e -> 1, Integer::sum))); System.out

Java Regex recursive match

喜欢而已 提交于 2021-02-19 07:34:07
问题 I have a string like this: IF(expr1,text1,IF(expr2,text2,IF(expr3,text3,text4))) I need to process IF(expr3,text3,text4))) so the above become IF(expr1,text1,IF(expr2,text2,new_text)) I need to keep doing this until I have IF(expr1,text1,new_text3) My pattern string "IF\\(.*?,.*?,.*?\\)" will match the whole thing. Not the IF(expr3,text3,text4))) . Java code: String val = "IF(expr1,text1,IF(expr2,text2,IF(expr3,text3,text4)))"; String regx = "IF\\(.*?,.*?,.*?\\)"; Pattern patt1 = Pattern

Liquibase rollback from command line not working

点点圈 提交于 2021-02-19 07:32:50
问题 I am doing an tomcat appliaction in a windows enviroment that when deployed creates/updates the DB Schema on the oracle db. For this I am using the Liquibase SDK 3.3.2. So basically I call the SDK an tell it to do an update from my changelog.xml. This parts works as fine. The code form the java class ... Liquibase liquibase = new Liquibase(CHANGE_LOG, new ClassLoaderResourceAccessor(getClass().getClassLoader()), db); liquibase.update(""); The problem is, when something goes wrong and I do a

How to verify Java Bytecode before injecting into the JVM?

孤人 提交于 2021-02-19 07:27:47
问题 I'm trying to verify "on the fly" generated bytecode! I already had several attempts, one attempt was to compile my classes in runtime with the eclipse compiler another was to compile from memory as mentioned here: Compile From Memory First results looked okay but I'm still not a 100% sure if the verification process is truly valid according to the JAVA 6 guidelines and security measurements as well to normal OOP Models. Any better way on how to verify bytecode? 回答1: If you load the generated

How to split a string into a map, grouping values by duplicate keys using streams?

北城余情 提交于 2021-02-19 07:24:27
问题 I want to convert the following String flString="view1:filedname11,view1:filedname12,view2:fieldname21"; to a Map<String,Set<String>> to get the key/value as below: view1=[filedname11,filedname12] view2=[fieldname21] I want to use Java 8 streams. I tried Arrays.stream(tokens) .map(a -> a.split(":")) .collect(Collectors.groupingBy( a -> a[0], Collectors.toList())); However the keys are also getting added to the value list. 回答1: You should use a Collectors::mapping to map the array to an

Passing array from java to dll function with JNA

为君一笑 提交于 2021-02-19 07:21:29
问题 I want to pass Java array as parameter to c dll throw JNA , here is my code : import com.sun.jna.*; public class Javatest { public interface CLibrary extends Library { CLibrary INSTANCE = (CLibrary) Native.loadLibrary( "test", CLibrary.class); void test(Pointer p,int width); } public static void main(String[] args) { Pointer p = new Memory(5*Native.getNativeSize(Double.TYPE)); for (int i = 0; i < 5; i++) { p.setDouble(i*Native.getNativeSize(Double.TYPE),5); } CLibrary.INSTANCE.test(p,5); } }

Java ClearCase Update on a HTTPS Server

我只是一个虾纸丫 提交于 2021-02-19 07:18:07
问题 I am trying to automate ClearCase, so I can do automatic building. The first step is to update a view by script. I am trying this with the following Java code: StpProvider provider = (StpProvider) ProviderFactory.createProvider( CcProvider.NETWORK_PROVIDER_CLASS, callback); provider.setServerUrl("https://..."); CcProvider m_provider = provider.ccProvider(); m_provider.registerTrustManagerCallback(callback); File viewRoot = new File("D:\\Views\\..."); StpLocation viewLocation = provider

Keep getting error that says Server App Engine Standard at localhost failed to start

别来无恙 提交于 2021-02-19 07:15:28
问题 When trying to start a google app engine project in eclipse i keep getting an error saying Server App Engine Standard at localhost failed to start(error log below). Has anyone come across this issue before and what was the cause of the issue. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.appengine.tools.development.StreamHandlerFactory (file:/C:/Users/AppData/Local/google/ct4j-cloud-sdk/LATEST/google-cloud-sdk/platform/google

eclipse says “already running”, but no big red “close” button

雨燕双飞 提交于 2021-02-19 07:14:28
问题 I am sure I would be able to find the answer if I knew the right terminology for some of these things, but basically the problem is: Every time I have run a something in Eclipse (Java) before, there has been a square red button (hovertext "Terminate") in the "Console" tab of the bottom-right (wide) window (in "Java" perspective) that I could use to terminate a program. I recently added a new project (to isolate and figure out some code I was having trouble with in the main project...but that